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I. INTRODUCTION 



A . BACKGROUND 

As seen in several of the recent wars and conflicts, 
speed and timing are crucial in modern warfare. In the 
Falklanda War, the lack of time available to react to a 
threat caused the loss of HMS SHEFFIELD. The HMS SHEFFIELD 
was sunk by fires that could not be brought under control as 
a result of a strike by an Exocet missile. Even though the 
ship had weapons systems that could have defeated the 
Exocet, its inability to initially detect the missile at a 
far range rendered these defenses useless. The Electronic 
Warfare <EW) operators on the SHEFFIELD had little warning 
of the Exocet due to self-induced jamming. When the self- 
jamming (inadvertent, of course) ceased, the Exocet was 
immediately detected, but it was too late to engage. The 

missile struck about ten seconds later. Although 
technologically superior, the British did not correctly 
manage the Radio Frequency <RF) spectrum and lost a ship. 
The self-jamming was caused by equipment interference and 
was either not noticed earlier or dismissed as unlikely to 
cause serious problems. This problem and others like it are 
now being resolved by the British. 

In war at sea today, it is necessary to provide an 
adequate reaction time. Reaction time is the time between 
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detection of the incoming target and weapons engagement. 
Response time is the time between enemy weapons release and 
impact (i.e.^ the time available for detection, reaction, 
and engagement). In the example cited above, that adequate 
reaction time simply was not present. In the case of HWS 
SHEFFIELD, improper management of the electromagnetic (EM) 
spectrum set up the situation of inadequate reaction time. 
With the coming of sophisticated weapon systems and 
supersonic missiles, the amount of time available to respond 
to a threat has been steadily reduced. In World War II 
reaction time could be measured on the order of dozens of 
minutes. Today reaction times are on the order of dozens of 
seconds. With initial detection at the horizon, sea- 
skimming missiles offer only 30 seconds warning before 
impact. Today's Combat Information Center (CIO needs to be 
organized in such a way as to derive maximum efficiency and 
speed from operations in order to reduce reaction time as 
much as possible. 

The problem of reduced reaction time is not new and 
equipments in many areas has been developed to meet this 
need. The Navy Tactical Data System (NTDS) was developed to 
solve this problem. It reduces the amount of time needed to 
understand the tactical environment around the ship and by 
providing a more complete picture, aids sound decision 
making . 
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With the development and installation of effective 
defensive weapon systems onboard Navy ships, effort must now 
be devoted to the reduction of reaction time. Effective 
weapons are available, only the time to employ them 
correctly is needed. Effective long range sensor systems 
can provide adequate warning and **buy“ time for the 
employment of the appropriate weapon. Therefore, it can be 
concluded that anything that "buys time** is of value. 

But how does one buy time? There are two ways: (1) 
machines can be built to react more quickly or, (2) 
operators can be trained to respond faster. Although 
systems will help the Fleet sailor react quicker, there is a 
limit as to how fast he can respond. Working spaces need to 
be optimized so that the sailor can respond optimally. In 
this context, efficiency translates into speed which 
translates into reduced reaction time. The efficient 
arrangement of equipment in a working space has not been 
addressed by the Navy in such a way as to promote effective 
and efficient spaces. (For the remainder of this effort, 
the term **working space** or **work space** will be used to 
denote a combat apace where data is searched for, collected, 
evaluated, disseminated, and/or acted upon.) An efficient 
and effectively laid out workspace will, intuitively, buy 
time. The barriers imposed by improper design and poor 
layout can never be totally compensated for by training. 
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B. NEED 



During the procurement cycle, there is a requirement to 
perform human factors engineering on all new equipment to 
insure an adequate man-machine interface. However, there is 
inadequate methodology for insuring that space arrangement 
contributes to successful mission accomplishment. Space 
arrangement is, apparently, dictated primarily by the need 
to fit new and existing equipment into a space. This is not 
intended to belittle the efforts of those who are charged 
with designing the layouts and arrangements of combat 
spaces, but is intended to address methods for improving the 
efficiencies of layouts. 

Before the layouts of combat spaces can be redesigned, 
it is first necessary to determine if there is a deficiency 
in the existing layout. A measure of adequacy of space 
arrangements must be developed. At present there is no such 
measure for combat systems layouts. 

The field of human factors engineering has developed 
techniques for assessing the adequacy of tasks, subsystems, 
systems, and organizations. However, due to the dynamic 
nature of shipboard work space development there appears to 
have been few human factors engineering techniques 
addressing the arrangement of systems as applied to the 
space as a whole or the entire mission work areas. This is 
due to the fact that new equipment with new functions and 
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increased capabilities are constantly being introduced into 
spaces barely adequate £or the original equipment. 

As a result of the procurement cycle, human factors 
engineering is applied only on single systems or consoles. 
It has been recognized that training personnel to overcome 
the human factors design deficiencies is not cost effective 
in terms of either time, money, or manpower. The current 
requirement is for total individual system analysis in the 
areas of compatibility, interoperability, and human factors. 
It is mainly in the area of integrating these systems 
together in a work space that significant improvement is 
needed. 

C. PURPOSE OF THESIS 

The purpose of this thesis is to present and discuss a 
method of assessing the effectiveness of a work space 
layout. In addition, this method will provide the framework 
for pinpointing those areas where redesign will be most cost 
effective . 
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II. THE NATURE OF THE DIFFICULTY 



A . CURRENT METHOD 

How are space layouts currently assessed in the United 
States Navy? 0r» perhaps, a better question is: is the 
current method o£ assessing space arrangements adequate? 
What is the current method? 

There are two methods o£ improving a space layout. The 
first method is by fleet inputs. There are no formal 
procedures as such. To initiate a design change, a request 
for change (no specific format) is submitted by the 
individual (or ship) via his chain of command to the 
appropriate engineering office within the Naval Sea Systems 
Command (NAVSEASYSCOM) . An engineer studies the proposal to 
see if it has merit. If it does, he forwards it within 
NAVSEASYSCOM and through the appropriate system command. 
From this point, if it is acceptable, it is forwarded to the 
Office of the Chief of Naval Operations (OPNAV) . OPNAV is 
the configuration manager for all platforms and makes the 
final decision on configuration or layout design changes. 
For example, during the Board of Inspection and Survey (BIS) 
Acceptance Trials for USS CARL VINSON (CVN-70), a strong 
case for redesigning the CIC arrangement was made by the 
commissioning crew. In particular, the cramped space of the 
EW Module on CARL VINSON was addressed and a solution 
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proposed. That solution apparently was forwarded to 
NAVSEASYSCOH and Space Naval Warfare Command and then to 
OPNAV because it has been incorporated in the layout of the 
EW Nodule on CVN-71 (See Figure 1) . 

The other method in improving space design is a mock-up 
approach. At various organizations, mock-ups are used to 
test the layout designs considered. These organizations are 
under contract to produce a specific kind of mock-up. The 
Naval Ocean Systems Center (NOSC) in San Diego does some 
mock-up work under the direction of NAVSEASYSCOH. At 
present they have a carrier Combat Information Center (CIO 
mock-up. It contains the Display and Decision portion which 
includes the Surface Warfare Module and the Air Warfare 
Nodule. It does not contain Detection and Tracking, the 
Electronic Warfare Module, or the Anti-Submarine Warfare 
Module. 

B. DEFICIENCIES 

The current method of improving the layout/arrangement 
design of some of the spaces on our surface ships, has four 
major shortcomings: (1) lack of user input, (2) lack of 
human factors engineering, (3) lack of a learning curve, and 
(4) no planning for growth. These deficiencies and the 
impact they have on the effectiveness of the space is now 
discussed . 
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Figure 1. EW Module Layout (Proposed) 
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1 . Lack o£ User Input, 



The current, method o£ layout improvement has little 
or no operator/user input. There is, perhaps, some user 
input as in the CARL VINSON CIC example cited above but this 
appears to be the exception rather than the rule. There is 
no formal method of submitting a design change through 
normal Navy channels to NAVSEA. This is a very serious 
deficiency because sketchy or total lack of fleet input is 
counter-productive. The design of a space by those who do 
not and will not be using it has a tendency to result in a 
far from optimal design. For example, a radio or “bitch 
box** that is frequently used is placed just out of reach. 
In the EW Module on USS CARL VINSON, the 12 MC (internal 
communications set) used to communicate with the Tactical 
Action Officer (TAO) and INTEL (among others) is placed such 
that the EW Supervisor and NTDS operator have to get up out 
of his seat to talk. There is, in fairness, a hand mike 
that can be attached but this has the undesirable aide 
effect of cluttering the workstation. 

2. Lack of Human Factors Engineering 

The only human factors engineering being employed is 
basic. This method has been characterized as “moving the 
furniture around**. This is done until there is an 
apparently workable solution. Again there is inadequate 
fleet or operator inputs to check the **new“ arrangement. It 
must be said that those employing this method are trying to 
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£ind an arranganen't t.hat facill'ta'tes an efficient and 



effective operation. This method is the best method 
currently available to accomplish this task, but still 
something is lacking. 

3. Lack of a Learning Curve 

However good the results of the mock-up method may 
be, there is no apparent learning curve in successive 
layouts. For example, the EW Module layout on USS AMERICA 
provided the necessary room for the activities of EW and 
gave the impression of smooth efficiency and competence. 
However,, on later aircraft carriers (most notably, the 
NIMITZ class) the EW Module arrangement is a regression and 
in nowise approaches the room and layout effectiveness found 
on AMERICA. If the NIMITZ class carrier EW Module layout 
was intended to be an improvement over AMERICA, it failed. 

4 . Wo plan for Growth 

The current method is deficient in terms of its 
potential for growth. Few designs provide room for 
expansion for either new equipment or modifications to older 
equipment. When new equipment is added, the space for it 
must come from someplace, even if that area has another 
function. Simply adding new equipment does not aid the 
operation of the overall work area and may even be 
counterproductive in that efficiency may be reduced. 

When lead units are designed and built, they are 
constructed with only existing equipment is mind (this is a 
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general rule and there are some exceptions) . There is some 
small amount o£ room £or expansion o£ capabilities but it is 
thought that the new equipment will replace older equipment 
and take up the same amount <or less) o£ room and £it into 
the same space. This thought does not take into account new 
missions £or the space with corresponding new equipment, new 
capabilities. and new space requirements. There£ore. one 
can readily see that new equipment must be added wherever 
there is room £or it. Sometimes the space where the new 
equipment is added is unsatis£actory £or the equipment and 
its operation. By way o£ example. USS CARL VINSON is a 
NIMITZ class aircra£t carrier and was built using 
essentially the same blueprints as the lead ship. The EW 
Module apace waa not changed even though the SLQ-17 and SSQ- 
82 (MUTE) . not yet procured when NIMITZ was designed and 
built. were slated £or CARL VINSON (and all carriers, 
eventually) (See Figure 2) . Only one equipment rack was 
removed to make way £or both equipments. The SLQ-17 console 
fits into the vacated equipment rack. This still le£t the 
computer unit (about one and a hal£ racks) and MUTE (which 
was designed and built wider than the standard rack) to be 
placed somewhere within the EW Module. The SLQ-17 computer 
rack was placed against the bulkhead in the middle of the 
space. At most, three operators could fit comfortably into 
the EW Module even though General Quarters manning calls for 
four operators, and MUTE waa placed outside the EW Module in 
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Combat Direction Center 
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Figure 2. USS CARL VINSON EV^/ Module 



a passageway within CIC that was heavily traveled. There 
was simply no room in the EW Module for both the EW 
operators and MUTE — one or the other had to go . 

In addition to the problems cited above, to 
accommodate the inclusion of all the equipment in the EW 
Module, some severe space economies had to be made. The 
layout now took on the appearance as shown in Figure 2. To 
allow some passage of operators and maintenance people among 
and around the equipment, a "straight line” layout was 
adopted. This had the sole advantage of allowing all the 
equipment possible to be place in the space. However, the 
question can logically be asked, "Does such an arrangement 
add or detract from the efficiency and effectiveness of 
space utilization in accomplishing the mission?" New 
equipment added to a space that was not designed for it may 
cause integration problems due to its intrinsic nature 
<i.e., in the equipment itself), its new location (e.g., the 
SLQ-17 computer rack), and reduced workspace <in our 
example, several racks where one used to be to the exclusion 
of another piece of equipment--MUTE) . 

The remainder of this thesis will be given over to 
attempting to find a workable solution to the problem of 
adequately designing a work space, in particular, an EW 
Module. As indicated earlier this is an area where the costs 
are in dollars and effort, but the payoff is in shorter 
reaction time and, ultimately, in ships and lives saved. 
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APPROACH 



A. IMPROVEMENT TO LAYOUTS 

The 8olut.ion to layout/axrangement improvement is 
neither simple nor straightforward. An improvement, 
however, can be found in a threefold approach to the 
problem. These are: (1) a ship class, module, mock-up at a 
land based laboratory, <2> fleet inputs added to it on a 
regular basis, and (3) a quantifiable measure that can be 
used to determine overall effectiveness and pinpoint problem 
areas. 

Establishing a class, module, mock-up at a land based 
laboratory makes good sense. Here, the results of several 
mock-ups can be stored and compared. Here, too, a "learning 
curve" can be established. What does not work for one class 
and module may never work, or it may work for another class 
ship and another module. The cost of mock-ups can be kept 
low. Mock-ups of new equipment entering the fleet can be 
sent to just one location and then incorporated into the 
design or redesign. Mock-ups of new ship classes can easily 
be done there. 

NOSC at San Diego seems to be a good place to have this 
mock-up facility for several reasons. First, experts there 
have already done some mock-up work and have a certain 
amount of experience in this area. Secondly, they are near 
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a good source of flee'b lnput.8 In San Diego. Once a mock-up 
was designed <or redesigned), NOSC could request, some fleet 
operators from one of the ships of that particular class and 
these operators could critique the mock-up and make 
suggestions for improvement. For added realism and 
additional inputs, a mock scenario could be played out by 
the operators on the mock-up. This has the added possible 
benefit of uncovering any oversight by either NOSC or the 
operators' critique. The two logical places for the mock-up 
site are Norfolk, Va. and San Diego, Ca. 

Fleet inputs in the design/redesign of the layout 
process is of the utmost importance. The fleet operators 
are the people who have to use the equipment and accomplish 
the mission within the space. They, from the benefit of 
several years individual and many years collective 
experience, will be able to note problems with the mock-up 
that the designers may have missed. Designers of single 
equipments tend to think of their equipment in isolation 
from all others. Layout designers are often not familiar 
with the operating characteristics of all the equipment. 
Fleet operators suffer from neither of these deficiencies. 
However, operators do have a bias toward doing things as 
they are currently done and may resist change. Nevertheless, 
they are still probably the best ones to evaluate the mock- 
up. 



24 



As seemingly complete as the combination of both 
laboratory mock-up and fleet input might be, there is one 
more area that needs to be covered. This is a quantitative 
assessment of the present layout and mock-up layouts. There 
are several reasons for this. First, a quantitative 
assessment of a present layout may indicate that it does not 
need improving or that the cost of improving the layout is 
not justified by the amount of improvement. Second, a 
quantitative assessment based in part on questionnaires to 
fleet operators may awaken thoughts of some inadequacy that 
was not present in the conscious memory but was tucked away 
in the recesses of the mind. Finally, a quantitative 
assessment is necessary to be able to compare functional 
layouts one to another. The final aspect of this approach is 
a way of assessing the effectiveness of the layout. 

Various techniques have been developed that will aid in 
assessing effectiveness. However, these methods have been 
used on systems that are dissimilar to those found on ships 
and must be modified. The method that will be utilized is a 
combination of three different but related techniques: Task 
Analysis, Link Analysis, and Operability Analysis. Two 
major studies have been reviewed to determine the extent of 
these analyses and how they might be modified for a layout 
improvement application. These are Integration Analysis and 
Mission Operability Assessment Technique (MOAT) . A brief 
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look at each of these will indicate the salient portions of 



each for this application. 

Integration Analysis is the integration of Task 
Analysis, Operator Interviews, and Link Analysis to evaluate 
a system's Functional Mock-up. Integration Analysis was 
designed as a viable Test and Evaluation technique for the 
earlier stages of Developmental Test and Evaluation (DT&E) 
in order to reduce design discrepancies and minimize 
acquistion costs and time CRef . 13. 

MOAT, an evaluation methodology, measures the 
operability of a system or subsystem in terms of operator 
tasks performed during a mission. It essentially is an 
Operability Analysis. 

In general, MOAT addresses the problem of how well an 
operator can use a system or subsystem to perform tasks 
within the mission context. Contrasted to evaluations 
using human engineering design criteria which present only 
pass or fail information, this technique provides 
information on the degree of system and/or subsystem 
success or failure . [Ref . 2:pp. 3-43 

The underlying techniques of task analysis, scaling 

methodology, and multi-attribute utility <MAU) theory have 

been integrated into one comprehensive methodology. 

MOAT systematically structures operator tasks in accordance 
with mission needs and then assesses the operability of 
these tasks through conjoint measurement. All assessments 
are then integrated within rules established through MAU 
theory. The output of MOAT is quantitative information 
about the operability of an entire system, such as a 
fighter or attack aircraft; the operability of specific 
subsystems such as radar, communications, or navigation, 
and; finally, the operability of each task performed during 
a mission phase. In short, MOAT measures subsystem and/or 
system goal attainment . CRef . 2;p. 43 
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Having briefly described both Integration Analysis and 
MOAT, parts of each were combined In this thesis to 
provide a technique that Is well suited to a layout 
improvement application. From MOAT the techniques to assess 
man-system compatibility (i.e.. Task Analysis and 
Operability Analysis) were used. This was considered in a 
larger context in order to assess man as a team rather than 
as an individual. Operability Analysis consists of two 
parts: multi-attribute utility <MAU) theory (to be discussed 
later), and scaling theory. The use of questionnaires and 
Link Analysis came from Integration Analysis. Note that 
both Integration Analysis and MOAT contain Task Analysis. 
The questionnaires serve two purposes. First, they focus 
attention on the problem areas of the design/arrangement. 
Secondly, the completed questionnaires support assessment of 
the layout effectiveness. Finally, the questionnaires form 
a link between the various analyses and operator inputs. 

MOAT was designed to assess man-system compatibility. 
The original MOAT used a construct that embodied the three 
most important divisions of the man-system compatibility- 
man, system, and mission and how all three interact during 
mission accomplishment. The difference considered here lies 
in the evaluation of a team of operators rather than just 
one man and the fact that a group of subsystems arranged in 
a particular manner is used to determine the operability. 
Hence, the three most important divisions of the man-system 
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compa'tlbillty are now t.he team, system, and mission. As can 
be easily seen, the basic premise o£ MOAT is unchanged in 
that the man-system compatibility is still being evaluated. 
In short, is the arrangement o£ equipment in a space "user 
friendly"? Note that even though an arrangement of 
equipment within a work space is being specifically 
addressed, man-system compatibility subsumes the equipment 
arrangement within the work space. The basic contention is 
that the best operators and best subsystems in a poorly 
designed space may be less effective than operators with 
less ability and a less capable system that is in a space 
that is well designed for mission accomplishment. 

MOAT uses the term operability to reflect how the man, 

system, and mission interact during mission accomplishment. 

By definition, operability reflects (1) the amount of 
effort required by the operator in task accomplishment, (2) 
the degree of subsystem technical effectiveness in aiding 
the operator in task accomplishment, and (3) how important 
the task is for mission success. CRef. 2:p. 19] 

This can be redefined slightly to indicate (1) the amount of 

effort required by the operators (team) in task 

accomplishment (task difficulty) , (2) the degree that 

equipment arrangement aids the operators (team) in task 

accomplishment (arrangement effectiveness), and (3) how 

important the task is for mission success (task 

criticality) . 
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TASK ANALYSIS 



The purpose for Task Analysis Is t.o determine those 
tasks and subtasks needed to successfully perforin the 
mission of the Module. Without specifying the tasks 
performed within the EW Module, It would be difficult, if 
not Impossible, to determine layout effectiveness, 
task/subtask Identification forms the basis of both the Link 
Analysis that Is discussed later and the MOAT technique of 
Operability Analysis. Each task and subtask that operators 
perform will be examined and fit Into the larger picture of 
module mission. The effort within the EW Module can be shown 
to be divided hierarchically: Module .mission, operator 
tasks, and operator subtasks. This hierarchy is divided in 
the following manner: the aggregate of the subtasks 
comprises the individual task and the aggregate of the 
individual tasks comprises the mission. 

The mission of the EW Module is to conduct Electronic 
Warfare which includes Electronic Warfare Support Measures 
<ESM) and Electronic Warfare Counter-Measures <ECM) . This 
entails attempting to deny any potential enemy the 
exploitation of the electromagnetic spectrum while 
preserving it for our own use. The EW Supervisor (EWS) is 
responsible for providing timely evaluated EW information, 
EW data, and EW control <to the rest of the battle group 
when so designated as EW Control Ship) . This is accomplished 
by three operators and three work stations (WLR-1, SLQ-17, 
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and NTDS) . The taaka and aubtaaka that are performed within 
the workapace are liated in Table 1. While the module 
miaaion delineatea the overall reaponaibility for the 
module, the operator taaka are the firat major aubdiviaion. 
Theae are the taaka that each operator muat accompliah at 
hia workatation in order to contribute to miaaion 
accompliahment . The operator aubtaak ia a further diviaion 
of the operator taaka. Theae aggregate together for 
ESM/ECM. Theae are liated in Table 1 and were drawn from 
varioua aourcea and confirmed by the EW Module personnel. 
Each workatation and, therefore, each operator haa a piece 
of the ‘'puzzle** and only by putting them together can any 
aenae be made out of the parta. In thia caae, aa so many 
others, the whole will be greater than the sum of the parts. 

Note in Table 1 that there are actually five positions 
liated: EW Supervisor, WLR-1 operator, SLQ-17 operator, NTDS 
operator, and Status Board keeper. During normal steaming 
conditions, one of the three position operators is also the 
EW Supervisor and, therefore, he has a dual role to play. 
Additionally, there is no Status Board keeper during normal 
steaming watches. During Condition One, General Quarters, 
the Module is manned with five people. Therefore, the Task 
Analysis considered the more complicated situation of 
General Quarters. 
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TABLE 1 



OPERATOR TASKS AND SUBTASKS 



EW Supervisor 

Task r 1.1 Direct ESM search 
Subtaska 

1.1.1 Assign search parameters to SLQ-17 

1.1.2 Assign search parameters to WLR-1 

1.1.3 Assign ESM sensor report responsibilities -- own ship 

1.1.4 Assign ESM sensor report responsibilities -- force 

1.1.5 Initiate manual ID request - ship 

1.1.6 Initiate manual ID request - force 

1.1.7 Monitor automatic correlations/aaaociationa , (SLQ-17) 

Task ; 1.2 Report/Disseminate EW Information 
Subtaska 

1.2.1 Report evaluated EW information 

1.2.2 Provide EW recommendations 

1.2.3 Update status board near NTDS console 

1.2.4 Brief /debrief embarked Airwings 

1.2.5 Navigation by passive EW 

Task : 1.3 Counter Hostile Environment 
Subtasks 

1.3.1 Promulgation of ECM employment criteria 
NTDS Operator 

Task ; 2.1 Collect and enter EW data into NTDS 
Subtaska 

2.1.1 Enter manual ID information into NTDS 

2.1.2 Enter manual ESM/NTDS track associations 

2.1.3 Perform triangulation of ESM bearing lines 

2.1.4 Enter EW fixes 

2.1.5 Advise operators of bearing resolution 

2.1.6 Evaluate externally reported ESM bearings 

Task ; 2.2 Report/Disseminate EW Information 
Subtask 

2.2.1 Report evaluated EW information 

2.2.2 Update status board near console 

SLQ-17 Operator 

Task : 3.1 Conduct ESM Search 
Subtask 

3.1.1 Monitor automatic correlations/associations 

3.1.2 Establish operating modes of SLQ-17 (ESM) 

3.1.3 Enter detection and response parameters (ESM/ECM) 

3.1.4 Monitor environment on NTDS console 

3.1.5 Evaluate displayed data 
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TABLE 1 



OPERATOR TASKS AND SUBTASKS (continued) 



Task.: 3.2 Report/Disseminate EW Information 
Subtaak 

3.2.1 Report evaluated EW information 

3.2.2 Provide ESM/ECM data to NTDS 

3.2.3 Monitor entry of EW data into NTDS 

3.2.4 Update status board 

Task : 3.3 Counter Hostile Environment 
Subtaak 

3.3.1 Engage targets with ECM 

3.3.2 Establish ECM operating Modes 

3.3.3 Assist in promulgation of ECM employment criteria 
WLR-1 Operator 

Task.: 4.1 Conduct ESM Search 
Subtasks 

4.1.1 Search assigned bands 

4.1.2 Analyze intercepted signals 

4.1.3 Check intercepts for images/harmonica 

4.1.4 Accurately DF intercepted signals 

4.1.5 Assist in evaluating ECM 

Task ; 4.2 Report/Disseminate EW Information 
Subtaska 

4.2.1 Provide ESM data to NTDS 

4.2.2 Report evaluated EW information 

4.2.3 Update status board near position 

4.2.4 Log all intercepts 

Task ; 4.3 EMCON 
Subtaaks 

4.3.1 Monitor EMCON 

4.3.2 Report violations of EMCON 

4.3.3 Log violations of EMCON 

4.3.4 Monitor MUTE 

EW Status Board 

Task : 5.1 Maintain Status Boards 
Subtaska 

5.1.1 Communicate with operators 

5.1.2 Advise operators of any information received 

5.1.3 Update status boards 
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LINK ANALYSIS 



Link Analysis is a technique that will provide the 
information needed to produce an acceptable arrangement of 
men and machines in a system CRef . 3:p. 2043 . The idea is 
that the •'best arrangement” can be found only by optimizing 
different types of links that are important in the 
particular system being designed. By way of definition, a 
link is a connection between <a) an operator and a machine 
or <b> two operators CRef. 3:p. 2043. These links may be 
visual (such as an instrument scan), functional or manual 
(hand to control), or verbal (communications). 
Inefficiencies are present when links are comparatively 
long, crossing one another, blocked, or outside optimal 
visual or reach envelopes. The links are produced from the 
task analysis and illustrate all the operator-required 
functional, visual, and communication tasks. Link Analysis 
can be applied to all scenarios involved during all 
operational and emergency conditions CRef. 3:p. 205 and 
Ref. 43. Link Analyses are normally of two types: panel 
layout and tactical compartment or multiple operator work 
area. With the development and procurement of individual 
subsystems (i.e., WLR-1, SLQ-17, etc.) a certain amount of 
panel layout link analysis has been done. However, little 
if any has been done on the combination of systems arranged 
in a workspace (in this example, the EW Module). Hence, the 
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Link Analysis will necessarily be of the latter type Ci.e., 
multiple operator work area) . 

The multiple operator work area type of link analysis is 
dependent on the correlation matrix. Beginning with the 
correlation matrix and an area layout, all interactions 
(links) required to perform a particular functional task are 
examined in terms of the frequency with which they occur and 
their criticality. If the criticality is assigned a 
numerical value, it may be multiplied by the frequency in 
order to obtain a weighted link value. The work area is 
overlaid with the weighted links permitting a picture of all 
the interactions taking place within the system being 
analyzed. The system design can then be modified to shorten 
the distance between the workstations that are connected by 
the weighted links CRef. 53. 

Figure 3 contains the correlation matrix for the EW 
Module in CARL VINSON. A correlation matrix is a figure 
that provides an indication of the links between two 
operators, positions, or between an operator and a position. 
Usually a criticality associated with the particular links 
is included in the matrix. In Figure 3, only the links of 
interest are listed. The figure is read by selecting the 
two entries for which links are desired and reading 
diagonally down from the top one and diagonally up from the 
bottom one until the intersecting diamond is reached. The 
diamond contains both the particular links between the two 
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Internal Communications 
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Figure 3. Link Correlation Matrix 



entries and the criticality of those links. The notation 
along the right side of the figure indicates what links are 
between each component of the module. A separate notation 
provides an indication of the criticality of that particular 
link: moat critical, critical, and least critical. The moat 
critical link is one that is absolutely essential to 
accomplish the mission. A critical link is one that 
prevents severe degradation in the accomplishment of the 
mission and the least critical link has a small impact. It 
remains only to multiply the links by a frequency of 
operation to obtain a Weighted Link Value. The Weighted 
Link Value will be obtained from the results of the Link 
Analysis Questionnaire. 

The type of links are communication, visual, and manual. 
The communication links are further subdivided into internal 
and external. The internal communication links are the 
voice interaction between operators while the external 
communication links are those voice and/or electronic links 
with other modules, persons, and/or platforms. The visual 
links can also be divided into internal and external. The 
internal are concerned solely with those links between the 
operator and his equipment or console. The external are 
those between the operator and other consoles and/or 
equipment. Similarly, the manual links, internal and 
external, are between the operator and his equipment or 
console and the operator and other equipment and/or 
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conaolea, reapecblvely . Since we have already conaidered 
that, link analyaia may have been done on individual systems 
and our concern ia for the multiple operator work area, we 
will not be concerned with the internal visual and internal 
manual links. This leaves ;)ust four links analyses to be 
done; the internal and external communications, the external 
visual, and external manual links. 

When the internal communication links are considered, it 
is noted that there are links between the EW Supervisor and 
both the WLR-1 and the SLQ-17 operators to promulgate 
orders. Next, there are links back to the EW Supervisor 
when one or the other have found either the signal of 
interest or something else that may be of Interest. There 
are also links between the WLR-1 operator and the 5L0-17 
operator. This last may be queries for information about 
their particular equipment set up or the passing of 
information to directing the other-' a search. The links 
between all three may be in the form of equipment status or 
failure. The internal communication links are shown in 
Figure 4. 

The external communications involve primarily the EW 
Supervisor with any of the following; TAO, INTEL (the 
Intelligence center), SURFACE, AIR, Trackers, ASW (Anti- 
Submarine Warfare) Module, SSES (Ships Signal Exploitation 
Space), FLAG, FWC (Force Weapons Coordinator), or SWC 
(Ship's Weapons Coordinator). 
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Figure 4. Internal Communications Links 




The extarnal viaual links involve all of the operators 
viewing the status boards and each other's workstation. 
These external visual links are important for the additional 
information they provide to the operator. The viaual 
reference to the other workstation builds an internal 
working model of the environment within the operator's mind 
thus enabling him to more quickly fit new data into the 
tactical picture and anticipate subsequent events. Without 
this interaction* effective Electronic Warfare control can 
not be attained . 

Figures 5 and 6 show the external visual links. Figure 5 
shows visual links between operators and Figure 6 indicates 
visual links between the operators and the status boards 
(there are only two status boards that can be clearly seen) . 

The final link that will be considered is the external 
manual link. Although there is no requirement for an 
operator to control more than one workstation, there are 
some external manual links that must be addressed. For 
example, all those equipments that are part of the EW Module 
for which there is no manning authorized will fall under 
this category. The AN/SSQ-82 MUTE is a prime example. MUTE 
is a monitor device that requires no manning and only 
cursory glances to ensure that it is functioning correctly. 
When adjustment is needed due to Force or ship EMCON 
changes, an external manual link for one of the operators 
takes place. Another example is the SLQ-17 UYK-20 computer. 
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Figure 5. External Visual Links - Operators 
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Figure 6. External Visual Links - Status Boards 



This equipment, is not adjacent to the SLQ-17 and one of the 
operators <normally the SLQ-17 operator) may need to reload, 
reboot, or reconfigure the system in the event of a casualty 
or normal operations. Figure 7 shows the external manual 
links. 

D. OPERABILITY ANALYSIS 

In the introduction to this section, it was stated that 
Operability Analysis was comprised of MAU and scaling 
theory. MAU is a Bayesian-oriented decision-making 
paradigm. There are three major aspects of the MAU model 
which are particularly important to this application. First, 
the basic structure principle in MAU is hierarchical 
decomposition. The mission is broken down into hierarchical 
grouping of tasks and subtasks. The model provides the 
structure and rules necessary to investigate and integrate 
the interrelationships of all these tasks and subtasks. 
Second, the definition of utility used in the MAU model 
allows for the optimum evaluation of alternatives which is 
dependent upon the selection of a single criterion. This 
means that multidimensional outcomes must be transformed 
into a single figure of merit such as utility, system worth, 
system effectiveness, or, as in this application, 
operability. Third, a scaling of the selected criterion. 
The scaling methodology used in this application, as in 
MOAT, was conjoint measurement. 
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Figure 7. External Manual Links 



Recall that, operability can be viewed aa a function of 
taak criticality, operator workload, and space 
effectiveness. Therefore, when considering each task front a 
operability standpoint, each task that is performed has some 
combination of these three dimensions. There is a 
difficulty in assessing the degree of each attribute and 
combining them into a meaningful measure of operability. 
Since this can not be assessed directly by objective 
methods, the scaling methodology of conjoint measurement was 
devised to assess space operability subjectively. The 
problem of scaling tasks in dimensions of criticality, 
frequency, and system effectiveness has been successfully 
solved by using objectively anchored rating scales 
[Ref. 2:p. 20]. Therefore, a similar rating scale procedure 
seemed suitable in this instance. 

The major difficulties involved with this approach are 
those of measuring the degree of operator effort (or watch 
section effort) and the layout effectiveness. This was to 
be expected, however, since not only were these different 
from any known previous study but also involved 
interactions on a higher scale than that experienced before. 
There is a substantial correlation in rating of task 
difficulty and subsystem effectiveness. The attempt to 
solve the rating scale problem is accomplished by dividing 
it into two separate ratings. On the F/A-IS program it was 
desirable to have two ratings; one with respect to pilot 
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worklood 



(PW) and one with respect to the technical 



effectiveness (TE) of the subsystem CRef • 2:p. 20] • A 
similar approach is attempted here with operator workload 
<0W) (it is assumed that this can be directly translated 
into watch section workload) and space effectiveness <SE) . 
This application of Operability Analysis is concerned with 
assessing the task criticality, the operator workload, and 
the apace effectiveness of a module layout. All are values 
on an ordinal scale. Two of these, space effectiveness and 
operator workload, need to be upgraded, through some 
methodology, to an interval scale in order to aggregate them 
over all tasks to achieve an overall measure of Module 
operability. To this end, conjoint measurement and its 
associated scaling procedures seemed suitable for a 
transformation to the desired unidimensional interval level 
scale. It is here that the delta method was employed. What 
conjoint measurement and the delta method do is allow 
separate rating of OW and SE (despite their mutual 
dependency) to be obtained and be combined in such a manner 
that a one-dimensional scale, having interval properties, is 
created- This scale might just as well be called the 
combined OW-SE scale. Rating on this scale can then be 
plugged into the MAU model. An assessment of the 
effectiveness of the EW Module layout with respect to a 
certain subtask can be determined. Aggregated together. 
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these will provide an indioetion of the overall 
ef fectiveneaa of the layout upon miaaion accomplishment. 

E . QUESTIONNAIRES 

Link Analysis, Task Analysis, and Operability Analysis 
will be completed by a series of two questionnaires. The 
first questionnaire (the Link Analysis Questionnaire) was 
targetted at Link Analysis and provided the frequency 
component for a completed Link Analysis. The second 
questionnaire (the Operator Subtask Questionnaire) confirmed 
the Task Analysis that went into building it and also 
provided the raw data needed to perform the Operability 
Analysis. From the Operability Analysis came the assessment 
of the Total Module Operability (TMO) . 

The questionnaires, as shown in Appendix A, were 
designed to do two things. First, the Link Analysis 
Questionnaire gave a general idea of the type and degree of 
deficiencies in the space in terms of link deficiencies. 
Second, it focused the operators^ attention on the 
arrangement of the work space and any deficiencies that were 
there. It was hoped that the questions brought into sharp 
relief the difficulties for which the operator unconsciously 
compensates during the mission. By highlighting these 
deficiencies through the Link Analysis series of questions, 
the detailing of them in the Operator Subtaak 
Questionnaires, hopefully, provided good human engineering 
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data with which to evaluate the Module Operability of the EU 
Module . 

The Link Analysis Questionnaire was designed to determine 
the frequency of the various links in the EW Module. 
Combined with the correlation matrix that indicates 
criticality, this questionnaire determined the weights of 
the various links. This weighting indicated the most 
heavily used links. This, in turn, can focus attention on 
deficiencies in these links. A possible example of this 
might well be the abnormally long internal communication 
link between the NTDS console and the WLR-1 position. 

The Operator Subtask Questionnaire contained a section 
requiring an assessment for each of the forty-nine subtasks 
delineated in Table 1. The subtasks were drawn from a 
variety of sources (including USS CARL VINSON Combat 
Direction Center doctrine) and verified by the EW operators 
prior to the administration of the questionnaire. This 
assessment was the culmination of the Task Analysis and the 
beginning of the Operability Analysis. In the Subtask 
Assessment, each subtask on the questionnaire was evaluated 
by the EW operators with regards to Operator Workload, Space 
Effectiveness, and Criticality of the Subtask to the overall 
mission accomplishment. 

The Operator Subtask Questionnaire also contained the 
Ranking Matrix, where combinations of the various degrees of 
Operator Workload (OW) and Space Effectiveness (SE) were 
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TABLE 2. RANKING MATRIX 

In t.he following maLrix, t.he blocks are ranked from 
best to worst <1 to 16). The lowest numbered block is the 
intersection of the best of the rows and columns. The number 
two block is next best, and so on. Note the arrows and the 
phrases associated with them. Design means the design of 
the layout or arrangement. 
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ranked. This was of the utmost importance since the 
conjoint measurement of the subtask assessment and Module 
operability depended upon it. Table 2 contains a blank 
Ranking Matrix. The EW personnel were asked to rank the 
intersections from best to worst for the “typical '* subtask. 
It was assumed that the rank order for the matrix would vary 
little from subtask to subtask. Helms found this to be true 
CRef. 2:p. 34 ]. This may have been the most difficult part 
of the questionnaire and the EW operators were forced to 
draw upon all their knowledge and previous experience in 
order to produce a rank order that was meaningful and 
replicable. This matrix. the intersection of two ordinal 
scales (OW and SE) . is part of conjoint measurement. The 
Ranking Matrix was expanded and an interval scale 
constructed via a linear expansion know as the delta method. 
This resulted in an interval scaling from 0 to 100 and was 
used to evaluate the total Module operability. The delta 
method of converting two of these ordinal scales to an 
interval scale is described in Appendix C. 

Using this interval scale, the intersection of any 
particular set of Operator Workload and Space Effectiveness 
values on the returned Operator Subtask Questionnaire gives 
a predetermined Operability Value between O and lOO. An 
Operator Workload value between one and four inclusive 
served to identify a column while a Space Effectiveness 
value identified a row. The intersection of the row with 
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the column indicated the assessment o£ that particular 
subtask by an operator. For every subtask, this Operability 
Value was obtained for each rater and the mean and standard 
deviation were calculated. This mean value represented the 
Operability Value for that subtask. 

The remaining ordinal scale is that of the Criticality. 
There was no attempt to convert this to an interval scale. 
Although operators' skills might vary, causing significant 
deviations in the ratings from rater to rater, there should 
be only one standard for the criticality of a subtask as it 
relates to mission accomplishment. This single measure of 
criticality was taken to be the mean of the criticality 
ratings. The Operability Value was multiplied by the 
criticality resulting in a Weighted Operability Value. A 
Weighted Deficit Value was computed as <100 - Operability 
Value) multiplied by the Criticality of the subtask. 
Whereas the Weighted Operability Value will give an 
indication of the "goodness" of the layout for a particular 
subtask, the Weighted Deficit Value gives an indication of 
how much improvement is required to optimize Module 
Operability for a particular subtask. 

The Link Analysis Questionnaire was given approximately 
one week before the Operator Subtask Questionnaire. It was 
hoped that the brief exposure to the first questionnaire 
increased the accuracy of the second. 
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IV 



RESULTS 



To teat, this application/ a suitable plat£orin was 
required. The Electronic Warfare Module on an U.S. Navy 
aircraft carrier was selected. The particular ship, USS 
CARL VINSON <CVN-70> , was chosen for three reasons: 
availability, accessibility, and familiarity. CARL VINSON 
had just returned from a seven month cruise and was in a 
stand-down period and, so, available. The ship's homeport, 
Alameda, Ca., was readily accessible for the test. Finally, 
the ship's layout was familiar enough to the test director 
to allow a minimum amount of time to be spent on the ship 
and, therefore, lessen the impact upon the ship's daily work 
and schedule. 

There were limitations to the scope of testing. First, 
the test was not done at sea which produced two limitations. 
In regards to Link Analysis, operator usage of the various 
links and the associated frequencies could not be monitored. 
This was considered to be a major limitation in regards to 
only the Link Analysis portion of the test. The 
compensation for this was the Link Analysis Questionnaire 
concerning the frequency of link usage. A minor limitation 
concerned the inability to observe the actual Subtasks and 
ascertain the criticalities under actual conditions. This 
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was compensated by the Operator Subtask Questionnaire, which 
was considered adequate. 

A further limitation was the small number of valid 
responses for the questionnaires. There were three valid 
responses for the Link Analysis Questionnaire, five for the 
Rank Ordering portion of the Operator Subtask Questionnaire, 
and from five to seven for the Subtask Assessment portion of 
the Operator Subtask Questionnaire. While these numbers are 
small from a statistical point of view, they can not be 
discounted. The limited sample size should be an inducement 
for further testing. Furthermore, the sample size for any 
aircraft carrier will never be much greater than about 
twelve due to manning levels. The sample size was seven due 
to leave and various schools but included the personnel with 
the most experience. In may be argued that not testing 
other platforms is a limitation. However, since no two EW 
Modules on U.S. aircraft carriers are alike, the lack of 
multiple testing is a moot question. 

The test was conducted in the EW Module of USS CARL 
VINSON (CVN-70) . The Module was used so that the personnel 
could refresh their memory with regards to the layout as 
they evaluated the subtasks in relation to the layout. 

The guidance given to the EW personnel before and during 
the test stressed that they could ask any question they 
wished of anyone they wished. They were encouraged to 
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confer with each other about the workload. 



effectiveness 



and criticality. 

A. LINK ANALYSIS 

The results of the Link Analysis were taken from the 
Link Analysis figures and from the Link Analysis 

Questionnaire. The questionnaire was produced from the Link 
Analysis figures and the Task Analysis in order to determine 
the frequency that these links were used. The EW operators 
on USS CARL VINSON were asked to estimate how many times 
during a standard eight (8> hour watch they utilized the 
links. The Link Analysis Questionnaire is listed in 

Appendix A and the results of the Link Analysis is shown in 
Table 3. 

1 . Link Analysis Figures 

The most critical links were assessed to be the 
communication links between operators and the visual links 
between positions. The criticality of the links were chosen 
to reflect mission accomplishment and the frequency of usage 
confirmed the criticality. There were four links considered 
in the Link Analysis: internal communication, external 

communication, external visual, and external manual. Of 
these four, the two most important links are the internal 
communications and external visual. This is because the 
external communication will generally involve only one 
operator (the EW Supervisor/NTDS operator) and there is 
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TABLE 3. LINK ANALYSIS BY POSITION 

Uelgnted 

Frequency Link Criticality Link Value 



Position and Tasks 
WLR-1 Operator: 

1. Talk/cosaunicate with the SLQ-17 operator? 

2. View the presentation on the SLQ-17 console? 

3. Talk/comunicate with the I'TTDS operator? 

4. View the presentation on the NTDS console? 

5. View the NTDS Status Board (SB)? 

6. View the WLR-1 Status Board? 

7. Update the '«A.R-1 Status Board? 

8. Check (yisually) the SLQ-17 cc«puter? 

9. Reboot, reset, or work with the ^Q-17 conputer? 
iZ. Check WJTE? 

11, Change any settings on ?1ITE? 



SLQ-17 Operator: 

L lalk/coKTiJunicate with the NTDS operator? 

2. View the presentation on the NTDS console? 

3. Talk/coflsnunicate with the ^R-1 operator? 

4. View the presentation on the WLR-1 console? 

5. View the VLR-1 Status Board? 

6. View the NTDS Status Board? 

7. Update tne NTDS Status Board? 

8. Check (yisually) the SLQ-17 conputer? 

9. Reboot, reset, or worn with the SLQ-17 conputer? 

10. Check NOTE? 

11. Change any settings on .1UTE? 



NTDS Operatcr/EW Superyisor: 

1. TalK/connunicate with tne SLQ-17 operator? 

2. View the presentation on the SLQ-17 console? 

3. *^alk/connunicate with the WLR-1 operator? 

4. View t"-.e presentation on the WLR-1 console? 

5. View the WLR-1 Status Board? 

S. View the NTDS Status Boarc? 

7. jpdote tne NTDS Status Board? 

8. Check (visually) the SLQ-17 coaouter? 

9. Reboot, reset, or work with the SLQ-17 computer? 

10. Check MUTE? 

11. Change any settings on hUTE? 

12- Coounicate outside the Module? 
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KEY: IC - Internal Connuni cat ions; £C - External Conoounications; 
EV - External Visual; EM - External Manual 
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requirement. £or manual links outside of ones own 
position. An external communications link example is the 
link between the EW Supervlsor/NTDS operator and the 
communications that enable him to communicate outside the 
Module. However, this requires that the operator rise from 
his seat to communicate. As a remedy, the NTDS operator 
uses a hand mike that hangs down near his console. This is 
a partial solution because he still needs to rise from his 
seat to select another station on the communication box. 
Additionally, the hand mike hanging so close to his console 
presents a clutter problem. 

Note that the communication and visual link between 
the NTDS operator <EW Supervisor) and the WLR-1 operator is 
the longest and partially blocked. The links between the 
WLR-1 operator and the NTDS operator and SLQ-17 operator are 
long, allowing him to view very little of the environment. 
The WLR-1 operator's visual links are very long and the 
parallax effect severely degrades his observation. Note the 
long link lines between the SLQ-17 and NTDS positions and 
the WLR-1 Status Board, and the WLR-1 operator and the NTDS 
Status Board. Finally, note the very long external visual 
links to the SSQ-82 MUTE and that they cross. MUTE is 
required to be checked periodically for faults or changes in 
the various monitor boxes. The distance is great enough 
between MUTE and the rest of the Module that only the WLR-1 
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operat.or can effectively monitor it. However, this requires 
considerable movement on the part of the ULR-1 operator. 

2 . Link Analysis Table 

The frequency of the various links were determined by 
the Link Analysis Questionnaire. The ideal way to determine 
link frequency is to count the actions/link interactions 
during the watch. Since this was not possible, the 
questionnaire approach was chosen. The Link Analysis is 
intended here to focus attention at the links that are used 
most often. The frequency of link usage is multiplied by 
the weight (criticality) of the link and an indication of 
its relative importance is determined. 

When the links associated with the WLR-1 operator 
are considered, it can be noted the longest links are the 
internal communication and external visual links between him 
and the SLQ-17 and NTDS positions. These links are also the 
most critical and the most frequently used. The average 
number of times the operator tries to view the NTDS console 
is 73.333. Yet this console is the furthest away (see 
Figures 4 and 5) . The WLR-1 operator communicates more with 
the NTDS operator for two reasons. Many times the NTDS 
operator is also the EW Supervisor. The fullest picture of 
the entire environment of surface, subsurface, and air 
contacts is present on the NTDS. The other large frequency 
usage is the visual links for the presentation on the SLQ-17 
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console. This console is only slightly closer than the NTDS 
console • 

In the case of the SLQ-17 operator, the first six 
entries in Table 3 are the ones with the greatest 
criticality and the highest frequency of use. The high 
criticality and frequency associated with checking the SLQ- 
17 computer is understandable since the SLQ-17 operator is 
specifically trained to know what to look for on the 
computer face. Note that the SLQ-17 operator views the 
presentation at the NTDS console much more than that at the 
WLR-1 position. It can be seen from Figure 5 that these 
external visual links between the SLQ-17 and the NTDS are 
much shorter than between SLQ-17 and the WLR-1. At the same 
time, the SLQ-17 operator communicates more with the WLR-1 
operator than with the NTDS operator. This suggests that 
the SLQ-17 operator gets a better picture of the environment 
from the NTDS but better information concerning the 
environment from the WLR-1. 

The NTDS operator/EW Supervisor are combined because 
many times the EW Supervisor will man the NTDS console for a 
major portion of the watch. This is necessary because all 
the external communications are at or near the NTDS console. 
Note the large frequency and high criticality associated 
with communications outside the Module (external 
communications link) . There appears to be a reversal of 
interaction between the NTDS operator/EW Supervisor and the 
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WLR“1 and SLQ-17 positions. He views the SLQ-17 console 
more than communicates with the operator but talks more to 
the WLR-1 operator than views the WLR-1 displays. Recall 
from Figures 4 and 5 that both the internal communications 
and the external visual links between NTDS and WLR-1 are 
very long. Additionally, note how much he looks at the WLR- 
1 Status Board even though it is the furthest away (Figure 
6 ) . 

The Link Analysis is important since it serves to 

indicate which links are long, important, and possibly 

$ 

overworked. As such it can be used as a starting point in 
the redesign of a layout by showing which links need to be 
reduced in length. The Link Analysis results should also 
support the results of the Operability Analysis. 

B. OPERABILITY ANALYSIS 

The Operator Subtask Questionnaire was divided into two 
parts: the Subtask Assessment and the Ranking Matrix. The 

Subtask Assessment was given first. The criteria for this 
evaluation and the test itself are given in Appendix A- 

The second half of the Operator Subtask Questionnaire 
was the Ranking Matrix. All returned valid rankings (n=5) 
were entered into the matrix and a mean determined for each 
block and the matrix numbered accordingly. The standard 
deviation was calculated in case of a tie- This matrix with 
the mean rank values and the standard deviation is 
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Illustrated in Table 4. The resultant rank matrix is shown 
in Table 5. 

Next this rank ordering was converted to an interval 
scale. This was done by reversing the order of the 
numbering so that the best of the Operator Workload and 
Space Effectiveness is #16 and the worst is #1 (see Table 
6) . Using this as a base. the delta method of linear 
expansion was used to determine an interval scale. See 
Appendix C for a brief description. example of the delta 
method, and the final work sheet for this application. 

Table 7 shows the result of the delta method which is the 
desired interval scale. The results of the delta method 
were normalized by dividing all the blocks by the highest 
value in the block; in this application it was 102. Table 8 
is the normalized interval scale for this application. 

The Operability Value was weighted (multiplied) by the 
mean assessed Criticality of that particular Subtask to 
derive the Weighted Operability Value. The Weighted 

Operability Value has the potential to range from an 
absolute minimum of O (0x1) to an absolute maximum of 500 
(100x5). The range noted was 14.14 to 418.87. 

The Weighted Deficit Value gives an indication of how 
much improvement is needed to optimize Layout Effectiveness 
for a particular Subtask. The greatest Weighted Deficit 
Value was 485.00 while the least was 28.08. The Weighted 
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TABLE 4 



MEAN RANK ORDER AND STANDARD DEVIATION 
FOR EACH RATING MATRIX CELL 



1 

1 


1 

9.6 1 


1 

4.6 1 


2.4 


1 

1 


1 


1 

1 


1 

1 


2.50 1 
1 


1.14 1 
1 


0.55 


1 

1 




1 

1 


1 

1 


1 

11.0 1 


1 

8.0 1 


5.2 


1 

1 


2.8 


1 

1 


1 

1 


2.55 1 
1 


1.58 1 
1 


1.10 


1 

1 


0.84 


1 

1 


1 

1 


1 

13.8 1 


1 

10.8 1 


8.8 


1 

1 


5.4 


1 

1 


1 

1 


1.64 1 
1 


1.10 1 
1 


1.10 


1 

1 


1.34 


1 

1 


1 

1 


1 

16 1 


1 

14.0 1 


12.2 


1 

1 


10.4 


1 

1 


1 

1 


1 

1 


1.22 1 
1 


2.17 


1 

1 


3.13 


1 

1 



I I 
I Mean Rank I 
I Standard Deviation I 

I I 



TABLE 5. RANK ORDER OF OPERATOR RATING MATRIX 



4 


1 

1 

1 


9 


1 

1 4 

1 


1 

1 2 
1 


1 

1 

1 


1 


1 

1 

1 


3 


1 

1 

1 


12 


1 

1 7 

1 


1 

1 5 

1 


1 

1 

1 


3 


1 

1 

1 


2 


1 

1 

1 


14 


1 

1 11 
1 


1 

1 8 
1 


1 

1 

1 


6 


1 

1 

1 


1 


1 

1 

1 


16 


1 

1 15 
1 


1 

1 13 

1 


1 

1 

1 


10 


1 

1 

1 



12 3 4 



OW 



60 



TABLE 6. FINAL RANK ORDER INVERTED FOR DELTA METHOD 
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TABLE 7. DELTA METHOD SOLUTION FOR 
OPERATOR SUBTASK RATING SCALE 
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TABLE 8. NORMALIZED INTERVAL SCALE 
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Deficit Value can range from 500 to 0. The larger the 
number, the greater the amount of improvement is needed. 

The Total Module Operability for this particular EW 
Module was computed to be 39.2 X. This computation is as 
follows. There were 49 Subtask evaluated. The summation of 
the criticalities of these Subtasks in order to accomplish 
the mission was 200.84. By assuming a perfect layout, we 
can multiply by 100 to obtain a maximum score of 20,084. 
Next the Weighted Operability Values were summed to obtain 
the actual score of the Module of 7872.31. When the actual 
score is divided by the maximum, an indication of the 
effectiveness of the layout is obtained. 

Table 9 contains an ordering of the Subtasks by 
cumulative weight. This was determined by dividing the 
Weighted Deficit Value by the optimum layout effectiveness 
to determine how much the deficit each Subtasks comprises. 
These were then ranked from moat to least. This table gives 
an indication of which Subtasks should be improved first in 
order to achieve the most coat effective approach to 
improving the Module. 

Table 9 contains the rank ordering by cumulative 
weights, the Subtask number, a brief description of the 
Subtask and its associated position, the operators polled 
with their evaluation of the Subtaak in terms of Operator 
Workload and Space Effectiveness converted to an interval 
scale, the operability value (the mean of the operators' 
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table 9. RANK ORDER OF SUBTASKS BY CUMULATIVE WEIGHT 
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evaluations), the deficit value <100 - operability value), 
the mean criticality, the cumulative weight or percentage of 
the total deficit that that particular Subtask comprises, 
and the total percentage. 
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V. DISCUSSION AND RECOMMENDATIONS 



There has been no at.t.empt. bo aacerbaln what Weighted 
Deficit Value or Weighted Operability Value is acceptable. 
This is beyond the scope of this effort. The purpose has 
been to identify which areas are in need of improvement and 
what areas should be addressed first in order to realize the 
greatest amount of improvement for a given effort. To 
answer the question of what Weighted Deficit or Operability 
Value is acceptable will call for additional research 
targeting the Subtasks individually to a greater detail than 
was attempted here. 

A. LINK ANALYSIS DISCUSSION 

The Link Analysis results show that there is only one 
position that might be considered acceptable in relation to 
the lengths of its links. This is the SLQ-17 position. 
This can be seen in part from the relatively good showing 
that the SLQ-17 console position had in comparison to the 
other two positions. The SLQ-17 operator can easily view 
what is displayed on the NTDS console and, without excessive 
movement, view the WLR-1 displays. He is within good 
viewing distance of the NTDS Status Board and his own SLQ-17 
computer. The viewing distance to the WLR-1 position and 
its associated Status Board are rather long, but still 
viewable. Because of its relatively good positioning in 
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relation to the rest of the Module^ SLQ-17 entries were much 



lower in Table 9. This would indicate that the layout 
actually promotes increased operator compensation since the 
other positions did not score as well. A score of 39.2^ is 
an indication of a poor the Module layout contributing to an 
increased operator compensation burden. Were the module 
layout better, the operators would have felt much better 
about the Module and the score would have been higher. 

B. OPERABILITY ANALYSIS DISCUSSION 

Several observations can be made from Table 9. First, 
the SLQ-17 appears to be the best position in the EW Module 
since its first entry is in twenty-first place in the table 
and most of the entries are at the bottom of the table. 
Almost 27 % of the possible improvements can be made in the 
first seven entries and these are just for the EW Supervisor 
and the WLR-1 operator. Note that the criticalities of 
these Subtaska are the highest. In other words, these 
Subtasks which are very critical are poorly supported by the 
layout, relative to the less critical Subtasks. Most of the 
lower criticalities are associated with Subtasks that have a 
relatively good layout effectiveness. 

It can be reasonably argued that Module Operability of 
39.2 % is not sufficient for an EW Module. What can not be 
argued is how much improvement is enough. Nor can it be 
extrapolated from this study what improvement a 
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rearrangement, can result in. However, it can be seen that 
improvement can be made in certain areas, as is indicated by 
careful perusal of Table 9. 

C. EXTRAPOLATION 

Further, this approach can be used for possible 
extrapolations. For example, comparing Figure 1 and Figure 
2, similarities are noted. They have the same arrangement 
of positions (i.e., from left to right, WLR-1, SLQ-17, and 
NTDS) . The positions are arranged in "straight line" type 
of layout. This resulted in a low Layout Effectiveness 
rating for USS CARL VINSON. It may be readily conjectured 
that another arrangement would work better, namely, a 
"crescent" shaped layout with the NTDS in between WLR-1 and 
SLQ-1 and the supervisor's position raised and directly in 
back of the NTDS operator. 

D. CONCLUSION 

It can be concluded that the present configuration of 
the EW Module on USS CARL VINSON does not result in an 
optimal utilization of this Module in terms of EW mission 
accomplishment. Further, there is a real need to assess the 
layout operability of the warfare modules onboard U.S. Naval 
combatants. This thesis has provided one way in which a 
measure of the effectiveness of a particular layout can be 
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determined. Although this wea a limited teat, Indlcatlona 
are that thla approach worka, and further teatlng Is 
warranted. 

Building a new layout la urged with the hopea that It 
may prove by teatlng to be better than the laat one, using 
the Link Analysis and Operability Analysis Illustrated In 
this work. What la significant and useful from the Link 
Analysis Is that any Improvement in layout design should 
probably start with ensuring that the critical links are not 
overly long or taxed beyond their limit. Any Improvement to 
the layout design should take Into account these critical 
links to reduce them to their optimum and any changes must 
not adversely affect the links since in that case any gain 
in layout design may be cancelled by a loss In link 
utilization. By conducting tests at landbased test sites, 
the risks of error are reduced. By the utilization of mock- 
ups and fleet inputs, the risks can be reduced even further. 
The result is a more effective layout enhancing mission 
accomplishment . 

E. RECOMMENDATION 

It is recommended that a land based test facility be 
established that would incorporate the ideas, 
recommendations, and methods indicated In this thesis as a 
first step to upgrading our combat workspaces. 
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APPENDIX A 



OPERATOR SUBTASK QUESTIONNAIRE 
GENERAL INSTRUCTIONS 

The purpose of this questionnaire is a subjective 
evaluation of the layout effectiveness of the EW Module for 
use in an algorithm to determine, in an objective sense, the 
effectiveness of the layout in accomplishing the mission of 
the Module. To do this there is a series of subtasks 
differentiated by operator that must be assessed in terms of 
operator workload per subtask, space effectiveness per 
subtask, and criticality of the subtask toward overall 
mission accomplishment. What is required is to make this 
assessment based on your experience and expertise. There is 
no time limit, you may ask questions of anyone you wish, and 
you should go and look at the Module to make sure of your 
answers especially if you are unsure of some of the 
questions concerning movements. There are no right or wrong 
answers, but try to be as precise as you can. A scenario, 
hopefully similar to your recent operations in the Sea of 
Japan, has been constructed. For each of the subtaska on 
the next page, mark with an •'X'*, the description that best 
describes the operator workload (OW) and the space 
ef f ecti veness (SE) - If the arrangement of the space has 
little or no effect on subtask accomplishment, then it would 
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rat.e the highest (4) . Conversely, if the layout or 
arrangement of the space negatively impacts subtask 
accomplishment, then it would rate a <1). Give your 
assessment of the criticality of the subtask in relation to 
the overall mission accomplishment. The descriptions of 
criticality, operator workload, and space effectiveness are 
listed on a separate sheet. 



SCENARIO 

This scenario begins with the assumption of the watch 
by a particular section 

They are the on-coming watch section in the EW module 
of a NIMIT2 class aircraft carrier that is steaming in the 
open ocean with six escorts. The escorts are one VIRGINIA 
class cruiser, two SPRUANCE class destroyers, one OLIVER 
HAZARD PERRY class frigate, one LOS ANGELES class submarine, 
and an oiler. There are heightened tensions world-wide with 
a probable confrontation between the two super-powers. 
There is a Soviet task group within 200 NM . The task group 
is comprised of a KIEV class aircraft carrier, a KIROV class 
cruiser, a SOVERMENYY class destroyer, two KRIVAK III class 
frigates, a SLAVA class destroyer, and three auxiliaries. 
Additionally, ECHO II, VICTOR III, and OSCAR class 
submarines are known to be in the area but unlocated for the 
past twelve hours. A Mod-KASHIN is the tattletale for the 
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Battle Group. Both forces are within range of Soviet air 
power • 

General Quarters is not set, but a heightened Condition 
III steaming watch is manned. There has been a momentary 
lapse of 400 Hz power and the NTDS is being reloaded. The 
SLQ-17 needs to be reloaded and reprogrammed. As the NTDS 
is brought on the line, the WLR-1 operator is told to 
recheck the past entries in his log and verify that they are 
still active. After 15 minutes, the WLR-1 operator reports 
that he has intercepted several new signals. One is an 
airborne mapping and reconnaissance radar. One appeared to 
be a brief intercept of a submarine radar. Another is an 
air search radar and the last is a missile acquistion radar - 

The NTDS air trackers report jamming on both long range 
and 3-D air search radars. 

The SLQ-17 alarms and displays hostile missile symbols 
from both the suspected direction of the SOviet task group 
and two angles 30 degrees either side of the task group - 

Deck Launched Interceptors are airborne within one 
minute . 

The EW operator at the NTDS console is entering ESM 
bearing lines and attempting to identify unknown contacts. 
The SLQ-17 operator shifts operation of the ECM portion to 
automatic as the TAO frees weapons. The WLR-1 operator is 
attempting to search the known hostile missile homing radar 
ranges to facilitate identification. General Quarters is 
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sounded. The TAO orders EMCON to be set for battle and the 
WLR"1 operator selects EMCON D on MUTE. A quick check of 
both the NTDS scope and that of the SLQ-17 indicates that 
the number and direction of the inbound unknowns do not 
match. The EW watch section tries to match the emerging 
identification from the WLR-1 and SLQ-17 to both the SLQ-17 
and the NTDS presentations. 

If you have trouble envisioning this scenario, recall 
the Sea of Japan operations on your last deployment and 
consider the signal environment and tactics you saw then. 
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OPERATOR WORKLOAD. SPACE EFFECTIVENESS, and CRITICALITY 



WORKLQAD/COMPENSATION/INTERFERENCE (Mental and Physical) 



( 1 ) 

Workload Extreae 
Coapensation Extrene 
Interference Extreae 



( 2 ) 

Workload High 
Coapensation High 
Interference High 



(3) 

Workload Moderate 
Coapensation Moderate 
Interference Moderate 



(4) 

Workload Low 
Coapensation Low 
Interference Low 



SPACE EFFECTIVENESS 



( 1 ) 

Inadequate Performance 
Due to Layout 



( 2 ) 

Adequate Performance 
Achievable; Layout 
Sufficient to Spec- 
ific Task 



(3) 

Layout Enhances Spe- 
cific Task Accomplish- 
ment 



(4) 

Layout Design 
Integrates 
Muitiole Tasks 



CRITICALITY : How important is it that the operator /team be 
able to perform this task as compared to the other 
tasks in successfully completing the mission? 

Scale Value: 



(1) Of very small importance . Ability to perform this 
task as compared to other tasks in this duty is unimportant, 
or almost unimportant^ in order to successfully complete the 
mission of the Module. 



(2) Of small importance . This task within this duty is 
less important than moat tasks required to successfully 
complete the mission of the Module. 



(3) Of moderate importance . This task within this duty is 
about as important as most tasks required to successfully 
complete the mission of the Module. 



<4) Of substantial importance . This task within this duty 
is more important than moat tasks required to successfully 
complete the mission of the Module. 

(5) Of extreme importance . This task within this duty is 
extremely important in order to successfully complete the 
mission of the Module. 
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EW Supervisor 



Task,: 1 . 1 Direct. ESM search 

Subtaaka 

1.1.1 Assign search parameters 

Operator Workload: 

< 1 ) 

Space Effectiveness: 

<1) 

Criticality: 

1.1.2 Assign search parameters 

Operator Workload: 

( 1 > 

Space Effectiveness: 

( 1 ) 

Criticality: 

1.1.3 Assign ESM sensor report 

Operator Workload: 

< 1 ) 

Space Effectiveness: 

( 1 ) 

Criticality: 

1.1.4 Assign ESM sensor report 

Operator Workload: 

( 1 ) 

Space Effectiveness: 

(1) 

Criticality: 



to SLQ-17 



<2) 


(3) 


(4) 


(2) 


(3) 


(4) 


to WLR-1 






<2) 


(3) 


(4) 


(2) 


(3) 


(4) 



responsibilities 




own ship 


(2) 


(3) 


(4) 




(2) 


(3) 


(4) 




responsibilities 


-- 


f orce 


(2) 


(3) 


(4) 




(2) 


(3) 


(4) 
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1.1.5 Initiate manual ID request - ship 



Operator Workload: 
Space Effectiveness: 
Criticality: 



Operator Workload: 
Space Effectiveness: 
Criticality: 



Operator Workload: 
Space Effectiveness: 
Criticality: 



Subtaaks 



Operator Workload: 
Space Effectiveness; 



<1) 


<2) 


<3) 


(4) 


<1) 


<2) 


<3) 


<4) 


request - force 




(1) 


(2) 


(3) 


(4) 


(1) 


(2) 


(3) 


(4) 


lorrelations/associations , 


<1) 


<2) 


(3) 


(4) 


<1) 


(2) 


(3) 


(4) 


late EW 


Information 




1 information 






(1) 


(2) 


(3) 


(4) 


(1) 


(2) 


(3) 


(4) 



Criticality : 
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1.2.2 Provide EW recommendations 



Operator Workload: 

(1) (2) (3) 

Space Effectiveness: 

<1) <2> (3) 

Criticality: 

1.2.3 Update status board near NTDS console 

Operator Workload: 

(1) (2) (3) 

Space Effectiveness: 

(1) (2) (3) 

Criticality: 

1.2.4 Brief /debrief embarked Airwings 

Operator Workload: 

(1) (2) (3) 

Space Effectiveness: 

(1) (2) (3) 

Criticality: 

1.2.5 Navigation by passive EW 

Operator Workload: 

(1) (2) (3) 

Space Effectiveness: 

(1) (2) (3) 

Criticality: 



Task : 1.3 Counter Hostile Environment 
Subtasks 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 
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1.3.1 Promulgate ECM employment 


criteria 




Ooerator Workload: 


<1) 


(2) <3) 


(4) 


Soace Effectiveness: 


<1) 


(2) <3) 


(4) 



Criticality: 



NTDS Operator 



Task: 2.1 Collect and 
Subtaaks 

2.1.1 Enter manual ID 


enter EW data into NTDS 
information into NTDS 




Operator Workload 


• 

m 






<1) (2) 


<3) 


(4) 


Space Effectiveness : 








<1) (2) 


<3) 


(4) 



Criticality: 

2.1.2 Enter manual ESM/NTDS track associationa 

Operator Workload: 

<1) (2) <3) <4) 

Space Effectiveness: 

<1) (2) <3) <4) 

Criticality: 

2.1.3 Perform triangulation of ESM bearing . lines 

Operator Workload: 

(1) (2) ( 3 ) <4) 

Space Effectiveness: _____ 

<1) ( 2 ) <3) <4) 

Criticality: 
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2.1.4 Enter EW fixes 



Operator Workload: 



Space Effectiveness: 


<1) <2) 


<3) 


<4) 


Criticalitv: 


<1) <2) 


<3) 


<4> 


2.1.5 Advise operators of 
Operator Workload: 


bearing resolution 




Space Effectiveness: 


<1) (2) 


<3) 


<4) 


Criticalitv : 


(1) (2) 


<3) 


( 4 ) 


2.1.6 Evaluate externally 
Operator Workload: 


reported ESM 


bearings 




Space Effectiveness : 


(1) <2> 


(3) 


(4) 


Criticalitv : 


(1) (2) 


(3) 


< 4 ) 


Task: 2.2 Reoort/Disseminate EW Information 
Subtaak 

2.2.1 Report evaluated EW information 
Ooerator Workload: 




Space Ef f ect i veness : 


(1) (2) 


<3> 


( 4 ) 


Criticalitv: 

2.2.2 Update status board 
Operator Workload: 


(1) (2) 
near console 


(3) 


< 4 ) 


Space Effectiveness : 


(1) (2) 


(3) 


< 4 ) 




<1) (2) 


(3) 


< 4 ) 



Criticality : 
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SLQ-17 Operator 



Task : 3.1 Conduct. ESM Search 
Subtaak 

3.1.1 Monitor automatic correlationa/aaaociationa 



Operator Workload : 



Space Effectiveness: 


(1) 


<2) 


<3> 


(4) 


Criticality: 


<i) 


<2) 


<3> 


<4> 


3.1.2 Establish operating 
Operator Workload: 


modea 


of SLQ- 


17 <ESM> 


Space Effectiveness: 




<2) 


(3) 


(4) 


Criticality : 


(1) 


<2) 


(3) 


<4) 


3.1.3 Enter detection and 
Operator Workload: 


response parameters 


(ESM/I 


Space Effectiveness : 


(1) 


<2) 


<3) 


(4) 


Criticality : 




<2) 


<3> 


(4) 


3.1.4 Monitor environment 
Operator Workload : 


on NTDS console 




Space Effectiveness : 


(1) 


<2) 


(3) 


(4) 




<1) 


(2) 


<3) 


(4) 



Criticality : 
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3.1.5 Evaluate displayed data 



Operator Workload : 



Space Effectiveness: 




<2) 


(3) 


Criticality ; 


<i) 


<2) 


(3) 


Task: 3.2 ReDort/Disseminate EW 
Subtask 


Information 


3.2.1 Report evaluated EW 
Operator Workload: 


information 




Space Effectiveness : 


<1) 


(2) 


(3) 


Criticality : 


(1) 


(2) 


(3) 


3.2.2 Provide ESM/ECM data 
Operator Workload: 


to NTDS 




Space Effectiveness: 


(1) 


(2) 


(3) 


Criticality : 


(1) 


(2) 


(3) 


3.2.3 Monitor entry of EW 
Operator Workload: 


data i 


nto NTDS 




Space Effectiveness : 


(1) 


(2) 


(3) 




(1) 


(2) 


(3) 



Criticality: 



(4) 



(4) 



( 4 ) 



( 4 ) 



( 4 ) 



( 4 ) 



( 4 ) 



( 4 ) 
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3.2.4 Update status board 

Operator Workload: 

<1) <2) (3) C4) 

Space Effectiveness: 

<1) <2) C3) (4) 

Criticality: 



Task : 3.3 Counter Hostile Environment 

Subtask 

3.3.1 Engage targets with ECU 

Operator Workload: 

<1) <2) (3) (4) 

Space Effectiveness: 

(1) <2) (3) (4) 

Criticality: 

3.3.2 Establish ECM operating Modes 

Operator Workload: 

(1) (2) (3) <4) 

Space Effectiveness: 

(1) (2) (3) (4) 

Criticality: 

3.3.3 Assist in promulgation of ECM employment criteria 

Operator Workload: 

<1) (2) <3) (4) 

Space Effectiveness: 

<1) (2) (3) <4) 



Criticality : 
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WLR-1 Operator 

Task : 4 . 1 Conduct ESM Search 
Subtaaka 

4.1.1 Search aaaigned banda 



Operator Workload : 



Space Effectiveness: 


(1) 


(2) 


(3) 


Criticalitv : 




(2) 


(3) 


4.1.2 Analyze intercepted 
Operator Workload: 


signals 




Space Effectiveness : 


(1) 


(2) 


(3) 


Criticality : 




(2) 


(3) 


4.1.3 Check intercepts f or 
Operator Workload : 


images/ harmonics 


Space Effectiveness : 


<i) 


(2) 


(3) 


Criticalitv : 




<2) 


(3) 


4.1.4 Accurately DF intercepted 
Operator Workload : 


signals 




Space Effectiveness : 


(1) 


<2) 


(3) 




(1) 


<2) 


(3) 



Criticality : 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 
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4.1.5 Assist in evaluating ECH 

Operator Workload: 

<1) (2) <3) 

Space Effectiveness: 

<1) <2) <3) 

Criticality: 



Task : 4.2 Report/Disseminate EW Information 

Subtasks 

4.2.1 Provide ESM data to NTDS 

Operator Workload; 

(1) (2) (3) 

Space Effectiveness: 

(1) <2) (3) 

Criticality: 

4.2.2 Report evaluated EW information 

Operator Workload: 

(1) <2) (3) 

Space Effectiveness: 

<1) <2) (3) 

Criticality: 

4.2.3 Update status board near position 

Operator Workload: 

<1) <2) (3) 

Space Effectiveness: 

<1) (2) (3) 

Criticality: 



<4) 



(4) 



(4) 



(4) 



(4) 



(4) 



(4) 



<4) 
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4.2.4 Log all int.ercepts 



Operator Workload: 



<1) 


<2) 


<3) 


(4) 


Soace Effectiveness: 









(1) <2) <3) <4) 



Criticality: 

Task : 4.3 EMCON 
Subtasks 

4.3.1 Monitor EMCON 



Operator Workload: 



Space Effectiveness: 


<1) 


(2) 


(3) 


(4) 


Criticality : 

4.3.2 Report violations of 
Operator Workload: 


(1) 

EMCON 


(2) 


(3) 


(4) 


Space Effectiveness : 


(1) 


(2) 


(3) 


(4) 




(1) 


(2) 


(3) 


(4) 



Criticality: 

4.3.3 Log violations of EMCON 
Operator Workload: 



(1) 


<2) 


<3) 


(4) 



Space Effectiveness: 

(1) (2) (3) <4) 

Criticality: 

4.3.4 Monitor MUTE 



Operator Workload: 



(1) 


(2) 


(3) 


(4) 


Space Effectiveness : 


(1) 


(2) 


<3) 


(4) 



Criticality : 



as 



EW Status Board 



Task : 5.1 Maint.ain Stat.us Boards 
Subtaaka 

5.1.1 Communicat-e with operators 



Operator Workload; 



Space Effectiveness; 


<i> 


<2) 


<3) 


(4) 


Criticality; 


<i) 


<2) 


<3) 


(4) 


5.1.2 Advise operators of 
Operator Workload; 


any 


information received 


Space Effectiveness : 


<i) 


(2) 


(3) 


<4) 




(1) 


<2) 


(3) 


(4) 



Criticality; 

5.1.3 Update status boards 



Operator Workload; 



<1) 


(2) 


(3) 


(4) 


Soace Effectiveness : 


(1) 


(2) 


<3) 


(4) 



Criticality : 
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Rat.lnQ Mat>rix Cell Rank Order 



For the following matrix, rank the blocks from best to 
worst <1 to 16) • The lowest numbered block is the 
intersection of the beat of the rows and columns. The number 
two block is next best, and so on. Note the arrows and the 
phrases associated with them. Design means the design of the 
layout or arrangement. Do not think of the layout of one 
workstation, such as the WLR-1 or SLQ-17, but of the entire 
EW Module. Think of the scenario already presented in order 
to properly consider the workload. Ask any questions you 
want or talk among yourselves or go and look at the layout. 



Multiple Tasks 
Integrated 



CP 

c 

•H 

> 

o 

u 

cu 

e 

H 



Layout Enhances 
Specific Task 
Accosiolishment 

Adequate 
Perfonftance 
Achievahle 
(Layout Sufficient) 



cn 

cn 

0) 

c 

(U 

> 

•H 

O 

0) 

M-l 

M-l 

w 



Inadequate 
Performance d-e to 
Layout, Cannot C:*;- 
pensate For SuC** 

Par Performance 



workload At 
Critical Levs:; 
Compensation 
Very Excessive 



k^rkioao Con- 
siderably Higher 
Than Antic: Dated; 
Ccr.csnsa:i:n 
Hi;h; Inte'^'^srenc? 
Major 



Worklcaa Siigntly 
higher Than 
Anticioatec; 
Moderate 
Cc^mper.sation; 

Miner Interference 



fvcrnleac As 
Anticioateci; 

No Interference; 
No Dofi'iDensation 



WORKLOAD IMPROVING 
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LINK ANALYSIS QUESTIONNAIRE 



WLR-1 Operator; 

How many times in an eight hour watch do you: 



1. Talk/communicate with the SLQ-17 operator? 



2. View the presentation on the SLQ-17 console? 



3. Talk/communicate with the NTDS operator? 



4. View the presentation on ■ the NTOS console? 



5. View the NTDS Status Board (SB)? 



6. View the WLR-1 Status Board? 



7. Update the WLR-1 Status Board? 



8. Check (visually) the SLQ-17 computer? 



9. Reboot, reset, or work with the SLQ-17 computer? 



10. Check MUTE? 



11. Change any settings on MUTE? 
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SLQ-17 Operat.or: 

How many timea in an eight, hour watch do you: 



1. Talk/communicate with the NTDS operator? 



2. View the presentation on the NTDS console? 



3. Talk/communicate with the WLR-1 operator? 



4. View the presentation on the WLR-1 console? 



5. View the WLR-1 Status Board? 



6. View the NTDS Status Board? 



7. Update the NTDS Status Board? 



8. Check (visually) the SLQ-17 computer? 



9. Reboot, reset, or work with the SLQ-17 computer 



10. Check MUTE? 



11. Change any settings on MUTE? 



NTDS Operat.or/EW Supervisor: 

How many -times in an eigh-t hour watch do you: 

1. Talk/communicate with the SLQ-17 operator? 

2. View the presentation on the SLQ-17 console? 

3. Talk/communicate with the WLR-1 operator? 

4 . View the presentation on the WLR-1 console? 

5. View the WLR-1 Status Board? 



6. View the NTDS Status Board? 



7. Update the NTDS Status Board? 



8. Check (visually) the SLQ-17 computer? 



9. Reboot, reset, or work with the SLQ-17 computer? 



10. Check MUTE? 



11. Change any settings on MUTE? 



12. Communicate outside the Module? 
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APPENDIX B 



QUESTIONNAIRE RESULTS 
Rating Ma~trix Cell Rank Order 
In the following matrix, the blocks rank from best to 
worst (1 to 16). The lowest numbered block is the 
intersection of the best of the rows and columns. The number 
two block is next best, and so on. Note the arrows and the 
phrases associated with them. Design means the design of the 
layout or arrangement. 



Multiple Tasks 


Integrated 


9 


6 




1 


Layout Enhances 


Specific Task 


12 


7 


4 


3 


flccoanpiisnnient 

Adequate 

Performance 

Achievable 


14 


11 


10 


c 

w 


(Layout Sufficient) 

Inadequate 
Perforwance due to 
Layout, Cannot Com- 


16 


15 


13 


d 


pensate For Sutr 
Par Performance 


Work load At 
Critical Level; 
Compensation 
Very Excessive 


Work load Con- 
sicerably Hirher 
Than Anticipated; 
Compensation 
High; Interference 
Major 


Workload Sligntiy 
Higner Than 
Anticipated; 
Moderate 
Conjcensaticn; 

Interference 


worn load As 
Anticipated; 

No Interfererce; 
No Ccsipensation 
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RESULTANT INTERVAL SCALE for OPERABILITY 



Multiple Tasks 


Integrated 


54 


75 


88 


100 


Layout Enhances 


Specific Task 


40 


61 


74 


85 


AccosplisPnent 

Adequate 

Perforaance 

Achievable 


24 


45 


58 


70 


(Layout Sufficient) 

Inadequate 
Perfomance due to 
Layout, Cannot Coa- 


0 


21 


34 


46 


pensate For Sub- 
Par Performance 


Worn load At 
Critical Level; 
Compensation 
Very Excessive 


Workload Corr 
sideraoly Hicher 
Than Anticipated; 
Cofflfensation 
High; Interference 
Major 


Workload Slightly 
Higher Than 
Anticipateo; 
Mocerate 
C3.:ipen5ation; 

Miner Interference 


Workload As 
Ant ici pat EC ; 

No Interference 
No Compensation 
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RESULTS OF ALL OPERATORS 



NOTES: The following reaulLa are the Operability Value of 
all reaponaea for each aubtaak. Generally, n equalled 7 for 
the aubtaaka, although there were some with only six 
responses. The Standard Deviation is that of the sample and 
not the population (i.e., the standard deviation was 
calculated using n vice n-1) . The Interval Scale shown was 
baaed on the Ranking Scale which the EW operators provided 
through the questionnaire. The Ranking (an ordinal) Scale 
was then converted to an interval scale by means of the 
Delta Method (a linear expansion) . The Criticality listed 
is a mean for the aubtask. The Weighted Operability Value 
is simply the Operability Value weighted (multiplied) by the 
Criticality . 

EW Supervisor 



Task : 1.1 Direct ESM search 
Subtasks 

1.1.1 Assign search parameters to SLQ-17 

Operability Value; 80.143 

Criticality: 3.167 

Weighted Operability Value: 253.81 

Weighted Deficit Value: 62.89 

1.1.2 Assign search parameters to WLR-1 

Operability Value: 49.714 

Criticality: 4.143 

Weighted Operability Value: 205.97 

Weighted Deficit Value: 208.33 
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1.1.3 Assign ESH sensor report, responsibilities -- own ship 



Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 



44 . 000 
4.500 

198.00 

252 . 00 



1.1.4 Assign ESM sensor report responsibilities 



Operability Value: 24.429 

Criticality: 4.000 

Weighted Operability Value: 97.72 

Weighted Deficit Value: 302.28 

1.1.5 Initiate manual ID request - ship 

Operability Value: 40.800 

Criticality: 4.250 

Weighted Operability Value: 173.40 

Weighted Deficit Value: 251.60 



1.1.6 Initiate manual ID request - force 



Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 



22.000 

4.000 

88.00 

312.00 



1.1.7 Monitor automatic correlations/associations. 



Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 



90.333 

3.667 

331.25 

35.45 



Task.: 1.2 Report/Disseminate EW Information 
Subtaaka 

1.2.1 Report evaluated EW information 



Operability Value: 3.000 

Criticality: 5.000 

Weighted Operability Value: 15.00 

Weighted Deficit Value: 485.00 

1.2.2 Provide EW recommendations 

Operability Value: 3.000 

Criticality: 5.000 

Weighted Operability Value: 15.000 

Weighted Deficit Value: 485.00 



- force 



(SLQ-17) 
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1.2.3 Updat.e s'tat.us board near NTDS console 



Operablllt.y Value: 60.143 

Criticality: 3.714 

Weighted Operability Value: 223.37 

Weighted Deficit Value: 14S.03 

1.2.4 Brief /debrief embarked Airwings 

Operability Value: 29.286 

Criticality: 3.286 

Weighted Operability Value: 96.23 

Weighted Deficit Value: 232.37 

1.2.5 Navigation by passive EW 

Operability Value: 17.500 

Criticality: 3.500 

Weighted Operability Value: 61.25 

Weighted Deficit Value: 288.75 



Task.: 1.3 Counter Hostile Environment 
Subtasks 

1.3.1 Promulgation of ECM employment criteria 



Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 



52.500 

4.500 
236.25 
213.75 



NTDS Operator 

Task : 2.1 Collect and enter EW data into NTDS 
Subtaaka 

2.1.1 Enter manual ID information into NTDS 



Operability Value: 9.857 

Criticality: 4.500 

Weighted Operability Value: 45.06 

Weighted Deficit Value: 405.64 

2.1.2 Enter manual ESH/NTDS track associations 

Operability Value: 20.286 

Criticality: 4.714 

Weighted Operability Value: 95.63 

Weighted Deficit Value: 375.77 
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2.1.3 Perform triangulation of ESH bearing lines 



Operabiliby Value: 25.000 

Criticality: 4.429 

Weighted Operability Value: 110.73 

Weighted Deficit Value: 332.18 

2.1.4 Enter EW fixes 

Operability Value: 22.143 

Criticality: 5.000 

Weighted Operability Value: 110.72 

Weighted Deficit Value: 389.29 

2.1.5 Advise operators of bearing resolution 

Operability Value: 23.714 

Criticality: 4.429 

Weighted Operability Value: 105.03 

Weighted Deficit Value: 337.87 



2.1.6 Evaluate externally reported ESN bearings 



Operability Value: 57.143 

Criticality: 4.286 

Weighted Operability Value: 244.91 

Weighted Deficit Value: 183.69 



Task : 2.2 Report/Dissemlnate EW Information 
Subtask 

2.2.1 Report evaluated EW information 



Operability Value: 34.714 

Criticality: 4.571 

Weighted Operability Value: 158.68 

Weighted Deficit Value: 298.42 

2.2.2 Update status board near console 

Operability Value: 51.714 

Criticality: 3.571 

Weighted Operability Value: 184.67 

Weighted Deficit Value: 172.43 



SLQ-17 Operator 

Task : 3.1 Conduct ESM Search 
Subtask 
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3.1.1 Monitor automatic correlations/asaociations 



Operability Value: 52.000 

Criticality: 4.286 

Weighted Operability Value: 222.87 

Weighted Deficit Value: 205.73 

3.1.2 Establiah operating modes of SLQ-17 (ESM) 

Operability Value: 80.571 

Criticality: 3.857 

Weighted Operability Value: 310.76 

Weighted Deficit Value: 74.94 



3.1.3 Enter detection and response parameters (ESM/ECM) 



Operability Value: 45.286 

Criticality: 4.429 

Weighted Operability Value: 200.57 

Weighted Deficit Value: 242.33 



3.1.4 Monitor environment on NTDS console 



Operability Value: 88.143 

Criticality: 4.000 

Weighted Operability Value: 352.57 

Weighted Deficit Value: 47.43 

3.1.5 Evaluate displayed data 

Operability Value: 46.000 

Criticality: 4.571 

Weighted Operability Value: 210.27 

Weighted Deficit Value: 246.83 



Task : 3.2 Report/Disserainate EW Information 
Subtask 

3.2.1 Report evaluated EW information 



Operability Value: 66.429 

Criticality: 4.286 

Weighted Operability Value: 284.71 

Weighted Deficit Value: 143.89 

3.2.2 Provide ESM/ECM data to NTDS 

Operability Value: 67.429 

Criticality: 4.000 

Weighted Operability Value: 269.72 

Weighted Deficit Value: 130.28 
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3.2.3 Monitor entry of EW data into NTDS 



Operability Value: 78.571 

Criticality: 2.571 

Weighted Operability Value: 202.00 

Weighted Deficit Value: 55.09 

3.2.4 Update status board 

Operability Value: 24.857 

Criticality: 3.571 

Weighted Operability Value: 88.78 

Weighted Deficit Value: 268.34 



Task.: 3.3 Counter Hostile Environment 
Subtask 

3.3.1 Engage targets with ECM 

Operability Value: 

Criticality: 

Weighted Operability Value: 

Weighted Deficit Value: 

3.3.2 Establish ECM operating Modes 

Operability Value: 71.714 

Criticality: 4.571 

Weighted Operability Value: 327.80 

Weighted Deficit Value: 129.30 

3.3.3 Assist in promulgation of ECM employment criteria 

Operability Value: 82.286 

Criticality: 3.333 

Weighted Operability Value; 274.26 

Weighted Deficit Value: 59.04 



88.857 

4.714 

418.87 

52.53 



WLR-1 Operator 

Task : 4.1 Conduct ESM Search 
Subtasks 

4.1.1 Search assigned bands 



Operability Value: 3.000 

Criticality; 5.000 

Weighted Operability Value: 15.00 

Weighted Deficit Value: 485.00 
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4.1.2 Analyze intercepted signals 



Operability Value: 14.286 

Criticality: 5.000 

Weighted Operability Value: 71.43 

Weighted Deficit Value: 428.57 

4.1.3 Check intercepts for inages/harmonics 

Operability Value: 87.714 

Criticality: 2.286 

Weighted Operability Value: 200.51 

Weighted Deficit Value: 28.09 

4.1.4 Accurately DF intercepted signals 

Operability Value: 11.286 

Criticality: 5.000 

Weighted Operability Value: 56.43 

Weighted Deficit Value: 443.57 

4.1.5 Assist in evaluating ECM 

Operability Value: 48.571 

Criticality: 2.286 

Weighted Operability Value: 111.03 

Weighted Deficit Value: 117.57 



Task : 4.2 Report/Disseminate EW Information 
Subtasks 

4.2.1 Provide ESW data to NTDS 



Operability Value: 3.000 

Criticality: 4.714 

Weighted Operability Value: 14.14 

Weighted Deficit Value: 457.26 

4.2.2 Report evaluated EW information 

Operability Value: 7.857 

Criticality: 5.000 

Weighted Operability Value: 39.29 

Weighted Deficit Value: 460.72 

4.2.3 Update status board near position 

Operability Value: 82.286 

Criticality: 2.429 

Weighted Operability Value: 199.87 

Weighted Deficit Value: 43.03 
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4.2.4 Log all int.ercept.s 

Operabilit.y Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 



Task : 4.3 EMCON 
Subtaaka 

4.3.1 Monitor EMCON 

Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 

4.3.2 Report violations of EMCON 

Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 

4.3.3 Log violations of EMCON 

Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 

4.3.4 Monitor MUTE 

Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 



EW Status Board 

Task.: 5.1 Maintain Status Boards 
Subtasks 

5.1.1 Communicate with operators 

Operability Value: 
Criticality : 

Weighted Operability Value: 
Weighted Deficit Value: 



13.429 

4.571 

61.38 

395.72 



46.429 

4.286 

198.99 

229.61 



44.143 

3.857 

170.26 

215.44 



49.286 

3.000 

147.86 

152.14 



35.571 

3.714 

132.11 

239.29 



10.573 
4.571 
48.33 
408 . 77 
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5.1.2 Advise operators of any information received 



Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 

5.1.3 Update status boards 

Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value:: 



24.429 
4.000 
97.72 
302 . 28 



20.289 
4.714 
95.64 
375 . 76 
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APPENDIX C 



THE DELTA METHOD 

This appendix gives a brief description of the delta 
method, an algorithm for converting ordinal measures on the 
cells of a matrix to a scale with interval properties 
satisfying the conditions of additive conjoint measurement. 
This appendix is essentially the same as that in Ref. 2, 
only being changed to reflect the present application. For 
a further and fuller description, see Coombs CRef . 63 . The 
method will be briefly described using the matrix in Figure 
C-1. This matrix is similar in form to the 4x4 matrix used 
in the OW/SE rating matrix developed for the Subtask 
Questionnaire but smaller in size. 





1 

R 1 
1 

1 


6 


1 

1 8 
1 
1 


1 

\ 

1 

1 


9 




Factor II 


1 

Q 1 

1 

1 


3 


1 

1 4 

1 
1 


1 

1 

1 

1 


7 






1 

P 1 
1 

1 


1 


1 

1 2 
1 
1 


1 

1 

1 

1 


5 





ABC 
Factor I 
Figure C-1 
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In Figure C-1^ Factors I and II represent two 
independent measures ^ and the numbers in the cells of the 
matrix represent an empirical ordering of overall 
performance over combinations of factors I and II. Higher 
numbers represent better overall performance. 

The resulting scales will be Interval measures of I and 
II as well as overall performance. Because the scale is 
additive^ the measure of overall performance of any cell 
must be the sum of the corresponding row and column scale 
values. Furthermore^ the resulting performance measure must 
reflect the ordering of the cells of the matrix. 
Consequently^ any set of scale values which provide an 
additive representation for a matrix must simultaneously 
satisfy the equations implied by the additive representation 
and the inequalities specified by the rank ordering of the 
cells of the matrix. Conditions under which a set of linear 
equations and inequalities have a common solution are 
specified mathematically by the Theorem of the Alternative. 
In practice, solutions may be found by using various linear 
programming techniques. The delta method is one such 
technique that is simple enough to be done by hand for small 
matrices . 

The delta method proceeds, in general, as follows. 
Cells in the matrix are initially given arbitrary positive 
scale values (represented by the Greek letter, delta; hence, 
the name) (we will replace delta with the Roman letter d, for 
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ease of computation) which satisfy the equations specified 
by the additive representation. For example, if B were 
assigned the value di and Q were assigned the value dg, then 
the cell BO would have the value dr^ dg. After initial 
assignments are made, the relationships between scale values 
and the d^s are changed to take into account the constraints 
given by the rank ordering of the matrix. When the 
procedure is completed and a solution is found, scale values 
are represented by positive linear combinations of the d's, 
such that any choice of positive d^'s will lead to scale 
values which satisfy both the equations implied by 
additivity and the inequalities implied by rank order. 

The levels of each factor are assigned values which 
reflect the ordering on that factor. Thus for Factor I, A 
will be assigned a value of O since it is the lowest level 
of the factor. B will be assigned an arbitrary positive 
value of di. Since C is a higher level of performance than 
B, it may be expressed as di > dg where dg is an arbitrary 
positive constant representing the difference between the 
scale values of B and C. Similarly, in Factor II, ?, G, and 
R may be assigned values of 0, dg, and d 4 , respectively. 
Since the overall performance is an additive combination of 
the two factors, the scale value of individual cells may be 
stated in terms of the d's as has been done in Figure C-2. 
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d3-»-d4 R 



Factor d 3 Q 
II 

0 P 



Alternately, the scale values may be displayed as in 
Figure C-3. this figure represents the work sheet which 
will be used in the algorithm- On the top half of the 

figure are the individual factor scale values- On th& 

bottom half are the values of overall performance, listed in 
decreasing rank order. The columns represent the four d's- 
the numbers in the work sheet are the coefficients in the 
equation : 

Scale Value = coefi x di '»• coef2 x d2 coefg x d 3 coef^ x 

d4 - 

A blank indicates a coefficient of zero- 



d3+d4 


1 

1 dX"'’d3-*-d4 

1 

1 


1 

1 di-»'d2*^d3-»'d4 

1 
1 


1 

1 

1 

1 


d3 


1 

1 di^-d3 

1 


1 

1 di + d2'^d3 

1 

1 


1 

1 

1 

1 


0 


1 

1 di 

1 

1 


1 

1 di-»-d2 

1 

1 


1 

1 

1 

1 


A 


B 


c 




0 


dl 


di-*-d2 





Factor I 



Figure C-2 
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A 

B 

C 

P 

Q 

R 

CR 

BR 

CQ 

AR 

CP 

BQ 

AQ 

BP 

AP 



The general procedure involves comparing cells adjacent 
in the rank ordering of the matrix and redefining the 
relationship between the scale values and the d's so that 



the order of the cells will be preserved for any choice of 



positive d's. 


the 


cells may be examined 


in any order; 


i n 


this 


example and 


the 


Layout Effectiveness 


application , 


we 


will 


start from 


the 


lowest performance and 


proceed to 


the 



highest level. This involves moving from the bottom to the 
top of the work sheet. 



The 


first pair of ceils 


in BP and AP. 


examination 


of 


Figure 


C-3 shows that BP is 


higher than AP. 


Since BP 


has 



di 



do d*P 



da 



Figure C-3 
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vQlue di and AP has value O, it may be concluded that di > 
O. This inequality is clearly satisfied for any positive 
value for di, so it is not necessary to redefine this value. 
It will be recalled that values were assigned to A and B so 
that B would have a higher value than A, 

The second inequality is AQ > BP. Substituting the 
values in this inequality gives dg > d^. This inequality is 
not true for all values of dg and di. However, since dg > 
di, it possible to replace dg by di ♦ ^3'# for positive dg^ , 
Now, for any choice of positive di and dg' , the inequality 
di dg' > di holds. On the work sheet, dg is replaced by 
di *»• dg' ; that is, in any row with a dg, a di and a dg' are 
added and the dg is deleted. For convenience, and because 
the dg' column looks exactly like the dg column, the dg' 
column is put where dg was. This is merely a relabeling of 
columns. Note that as many 'marks' as were in the dg column 
are added to the di column. The work sheet at this point 
looks like Figure C-4. 

The next inequality, BQ > AQ implies 2di > dg' > di > 
dg' or di > O. Again it is not necessary to make any 
changes to satisfy this constant. 
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di d<i 



A 


1 


1 


1 


1 


1 


1 


1 


B 


1 1 


1 


1 


1 


1 


1 


1 


C 


1 1 


1 1 


1 


1 


1 


1 


1 


P 


1 


1 


1 


1 


1 


1 


1 


Q 


1 1 


1 


1 1 


1 


1 


1 


1 


R 


1 1 


1 


1 1 


1 1 


1 


1 


1 


















CR 


1 11 


1 1 


1 1 


1 1 


1 


1 


1 


BR 


1 

111 


1 

1 


1 

1 1 


1 

1 1 


1 

1 


1 

1 


1 

1 


CQ 


1 

1 11 


1 

1 1 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


AR 


1 

1 1 


1 

1 


1 

1 1 


1 

1 1 


1 

1 


1 

1 


1 

1 


CP 


1 

1 1 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


BQ 


1 

1 11 


1 

1 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


AQ 


1 

1 1 


1 

1 


1 

I 1 


1 

1 


1 

1 


1 

1 


1 

1 


BP 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


AP 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 



AQ>BP=d3>di 

,d3 = di-^d3' 
BP>AP=di>0 

do nothing 



Figure C-4 

Proceeding up the work sheet, the next inequality is CP 
> BQ. This implies that d2 > di • Here, we do the 
same as before, in that we redefine d2 di d 3 ' d2' f 
and consequently replacing d2 in every row in which it 
occurs by a di, ci 3 ' , and d2^ • Again d2' is put in the 
column where d2 was. Consequently, the procedure involves 
relabeling the d2 column d2^ snd adding a di and a dg' for 
each d2 in any row in which a d2 appears. The work sheet at 
this point looks like Figure C-5. 
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di da 



A 


1 


1 


1 


1 


1 


1 


1 


B 


1 1 


1 


1 


1 


1 


1 


1 


C 


1 11 


1 1 


1 1 


1 


1 


1 


1 


P 


1 


1 


1 


1 


1 


1 


1 


Q 


1 1 


1 


1 1 


1 


1 


1 


1 


R 


1 1 


1 


1 1 


1 1 


1 


1 


1 



CR 


1 111 


1 1 


1 11 


1 1 


1 


1 


1 




BR 


1 

1 11 


1 

1 


1 

11 


1 

1 1 


1 

1 


1 

1 


1 

1 




CQ 


1 

1 111 


1 

1 1 


1 

1 11 


1 

1 


1 

1 


1 

1 


1 

1 




AR 


1 

1 1 


1 

1 


1 

1 1 


1 

1 1 


1 

1 


1 

1 


1 

1 




CP 


1 

1 11 


1 

1 1 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


CP>BQ=d2>di +d3 ' 


BQ 


1 

1 11 


1 

1 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


d2=di+d3' •►d2' 
BQ>AQ=di>0 


AQ 


1 

1 1 


1 

1 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


do nothing. 
AQ>BP=d3>di 


BP 


1 

1 1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


d3=di •►ds ' 
BP>AP=di>0 


AP 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


1 

1 


do nothing. 



Figure C-5 



The inequality AR > CP implies a similar inequality 
among d's and is handled the same way. The work sheet at 
this point is given in Figure C-6. 

The ordering of CQ > AR implies an inequality among tne 
d's which is handled somewhat differently from the previous 
inequalities. CQ > AR implies that di > d 3 ^ > d 4 ^. Since 

there are two d's on the left side of the inequality, it is 
not possible simply to make the substitution: 
di d3' = d4^ + di' + d3' ^ . 

Some rows may have different numbers of di and d 3 ', so this 
replacement rule would be impossible to implement. 
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The following t.hree at.ep met.hod is used t.o redefine t.he 



d's. 


First, d 4 ' is 


split into two parts. 


d 4 ' ' and 


d5. 


Since 


di ♦ dg' > d4' , 


this division may be 


arbitrarily 


done 


so that di > d 4 ' ' 


and dg' > ds. The 


preceding 


two 



inequalit.ies may be handled by 





di 


dy' 




da' 








A 


1 


1 


1 


1 1 


1 


1 




B 


1 1 


1 


1 


1 1 


1 


1 




C 


1 11 


1 1 


1 1 


1 i 


1 


1 




P 


1 


1 


1 


1 1 


1 


1 




Q 


1 1 


1 


1 1 


1 1 


1 


1 




R 


1 11 


1 1 


1 1 


1 1 1 


I 


1 




















CR 


1 nil 


1 11 


1 11 


1 1 1 


1 


1 




BR 


1 

1 111 


1 

1 1 


1 

1 1 


1 1 
1 1 1 


1 

1 


1 

1 




CQ 


1 

1 111 


1 

1 1 


1 

1 11 


1 i 

1 1 


1 

1 


1 

I 




AR 


1 

1 11 


1 

1 1 


1 

1 1 


1 1 
1 1 1 


1 

1 


1 

1 


AR>CP=d4>di *d2' 




1 


1 


1 


1 1 


1 


1 


d4 = d 1 *d2 ' +<34 ' 


CP 


1 11 


1 1 


1 1 


1 1 


1 


1 


CP>BQ = d2>di-^d3' 




1 


1 


1 


i 1 


1 


1 


d2 = d 1 <-d3 ' *d2 ' 


BQ 


1 11 


1 


1 1 


1 1 


1 


1 


BQ>AQ=di>0 




1 


1 


1 


1 1 


1 


1 


do nothing. 


AQ 


1 1 


1 


1 1 


1 1 


1 


1 


AQ>BP=dg>di 




1 


1 


1 


1 1 


1 


1 


d3=di -^d3 ' 


BP 


1 1 


1 


1 


1 1 


1 


1 


BP>AP=di>0 




! 


1 


1 


1 1 


1 


1 


do nothing . 


AP 


1 


1 


1 


1 i 


1 


1 





Figure C-6 



previously discussed methods. Thus the three replacements 
are : 

d4' = d4' ' * d5 , 

di = d4' ' * dl'* and 

d3' = ds' ' ♦ d5 . 
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Now for any choice of positive d's, CQ > AR is 
satisfied. The work sheet now looks like Figure C-7. 

The methods of handling the remaining inequalities have 
already been discussed. Note that the steps to complete the 
worksheet is (matrix size - 1), or in the example, 8. For 
the Module Operability application there were 15 steps. 

When completed, the top half of the work sheet shows the 
relationship between scale values and the newly defined d's. 
For the example given, the following relationships hold: 

A = 0 



B = 
C * 
P = 
0 = 
R = 


di' - 
2di' * 
0 

d±' + 
2di' ♦ 

di 


d3" 

d2' 

2d3' 

d2' 

dy 


- d4' 

* 3dg 

' * d4 

f 4dg 

' d?i ' 


^ A 

" - 2d4" 
♦ d5 

'' 3d4'^ 

d4' d-^ 




^ d5 
♦ 2ds 


A 


1 


1 


1 


1 


1 


1 


1 




B 


1 1 


1 


1 


1 


1 


1 


1 




C 


1 11 


1 1 


1 1 


1 11 


1 1 


1 


1 




P 


1 


1 


1 


1 


1 


1 


1 




Q 


1 1 


1 


1 1 


1 1 


1 1 


1 


1 




R 


111 


1 1 


1 1 


1 111 


1 11 


1 


1 






















CR 


1 1111 


1 11 


1 11 


1 11 


1 5 


1 


111 1 




BR 


1 

1 111 


1 

1 1 


1 

1 1 


1 1 
1 1111 1 11 


1 

1 


1 

1 


(d4'=d4' ' -"C15 




1 


1 


1 


1 


1 


1 


1 


(di=d4' ' ■►di ' 


CQ 


1 111 


1 1 


1 11 


1 111 


1 11 


1 


1 


CQ>AR=di+d3' >d4' 




1 


1 


1 


1 


1 


1 


1 


(dg' =d5*d3' ' 


AR 


1 11 


1 1 


1 1 


1 111 


1 11 


1 


i 


AR>CP = d4>dj_ +d2 ' 




1 


1 


1 


1 


1 


1 


1 


d4 = d 1 +d2 ' ■^d4 ' 


CP 


1 11 


1 1 


1 1 


1 11 


1 1 


1 


1 


CP >BQ = d2>d 1 +d3 ' 




1 


1 


1 


1 


1 


1 


1 


d2=di +dg' +d2 ' 


BQ 


1 11 


1 


1 1 


1 11 


1 1 


1 


1 


BQ>AQ=di>0 




1 


1 


1 


1 


1 


1 


1 


do nothing. 


AQ 


1 1 


1 


1 1 


1 1 


1 1 


1 


1 


AQ>BP=d3>d 1 




1 


1 


1 


1 


1 


1 


1 


d3=di>d3' 


BP 


1 1 


1 


1 


1 1 


1 


1 


1 


BP>AP=di>0 




1 


1 


1 


1 


1 


1 


1 


do nothing. 


AP 


1 


1 


1 


1 


1 


1 


1 





Figure C-7 
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Any choice of positive d's will give a scale for Factors I 
and II as well as overall performance which is an additive 
representation, and in which the overall ordering agrees 
with the empirical ordering. Figure C-8 contains the 
results of this example. 
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1 
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Figure C-8 



A common choice of d's is to make them all equal to 1. 
This choice for d yields a set of scale values which 
represent the set of minimal integers which will produce the 
requires rank order of the matrix cells. 

The completed work sheet for the Module Operability 
application follows. Note that the numbers within the d's 
are expressed in arable numerals for viewing ease and the 
totals are listed on the aide. 
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di ' 


" d'>" 




da' 


' ds' 


" da' 


" d7 


dA 




A 


1 


1 


1 


1 


1 


1 


1 


1 


1 = 0 


B 


12 


1 3 


1 4 


1 3 


1 3 


1 1 


i 4 


1 2 


1 = 22 


C 


13 


1 5 


1 7 


15 


1 4 


12 


16 


1 3 


1 = 35 


D 


14 


1 6 


1 10 


17 


16 


1 2 


1 8 


1 4 


1 = 47 


P 


1 


1 


1 


1 


1 


1 


1 


1 


1 = 0 


Q 


12 


1 3 


15 


1 4 


13 


1 1 


1 4 


1 2 


1 = 24 


R 


13 


15 


18 


1 6 


15 


1 2 


17 


1 4 


1 = 40 


S 


1 5 


1 7 
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18 


17 


13 


19 


1 5 


1 = 55 
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1 17 


1 9 
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Id 
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15 
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18 
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1 11 
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1 7 
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RC 
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1 13 
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QD 


16 
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19 
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1 = 71 


RB 


1 5 
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1 6 


1 2 


18 


1 4 


1 = 47 


QB 


1 4 


16 


1 9 


1 7 


1 6 


1 2 


1 8 


1 4 


1 = 48 


RA 


1 3 


1 5 
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1 3 
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1 2 
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PB 
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1 3 
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1 1 


1 4 


1 2 


1 = 22 


PA 


1 


1 


1 


I 


1 


1 


1 


1 


I = 0 



Figure C-9 
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